Jump to:  OMake Home • Guide Home • Guide (single-page) • Contents (short) • Contents (long)
Index:  All • Variables • Functions • Objects • Targets • Options

Chapter 3  Additional build examples

Let’s explain the OMake build model a bit more. One issue that dominates this discussion is that OMake is based on global project analysis. That means you define a configuration for the entire project, and you run one instance of omake.

For single-directory projects this doesn’t mean much. For multi-directory projects it means a lot. With GNU make, you would usually invoke the make program recursively for each directory in the project. For example, suppose you had a project with some project root directory, containing a directory of sources src, which in turn contains subdirectories lib and main. So your project looks like this nice piece of ASCII art.

    my_project/
    |--> Makefile
    `--> src/
         |---> Makefile
         |---> lib/
         |     |---> Makefile
         |     `---> source files...
         `---> main/
               |---> Makefile
               `---> source files...

Typically, with GNU make, you would start an instance of make in my_project/; this would in term start an instance of make in the src/ directory; and this would start new instances in lib/ and main/. Basically, you count up the number of Makefiles in the project, and that is the number of instances of make processes that will be created.

The number of processes is no big deal with today’s machines (sometimes contrary the the author’s opinion, we no longer live in the 1970s). The problem with the scheme was that each make process had a separate configuration, and it took a lot of work to make sure that everything was consistent. Furthermore, suppose the programmer runs make in the main/ directory, but the lib/ is out-of-date. In this case, make would happily crank away, perhaps trying to rebuild files in lib/, perhaps just giving up.

With OMake this changes entirely. Well, not entirely. The source structure is quite similar, we merely add some Os to the ASCII art.

    my_project/
    |--> OMakeroot   (or Root.om)
    |--> OMakefile
    `--> src/
         |---> OMakefile
         |---> lib/
         |     |---> OMakefile
         |     `---> source files...
         `---> main/
               |---> OMakefile
               `---> source files...

The role of each <dir>/OMakefile plays the same role as each <dir>/Makefile: it describes how to build the source files in <dir>. The OMakefile retains much of syntax and structure of the Makefile, but in most cases it is much simpler.

One minor difference is the presence of the OMakeroot in the project root. The main purpose of this file is to indicate where the project root is in the first place (in case omake is invoked from a subdirectory). The OMakeroot serves as the bootstrap file; omake starts by reading this file first. Otherwise, the syntax and evaluation of OMakeroot is no different from any other OMakefile.

The big difference is that OMake performs a global analysis. Here is what happens when omake starts.

  1. omake locates that OMakeroot file, and reads it.
  2. Each OMakefile points to its subdirectory OMakefiles using the .SUBDIRS target. For example, my_project/OMakefile has a rule,
        .SUBDIRS: src
    

    and the my_project/src/OMakefile has a rule,

        .SUBDIRS: lib main
    

    omake uses these rules to read and evaluate every OMakefile in the project. Reading and evaluation is fast. This part of the process is cheap.

  3. Now that the entire configuration is read, omake determines which files are out-of-date (using a global analysis), and starts the build process. This may take a while, depending on what exactly needs to be done.

There are several advantages to this model. First, since analysis is global, it is much easier to ensure that the build configuration is consistent–after all, there is only one configuration. Another benefit is that the build configuration is inherited, and can be re-used, down the hierarchy. Typically, the root OMakefile defines some standard boilerplate and configuration, and this is inherited by subdirectories that tweak and modify it (but do not need to restate it entirely). The disadvantage of course is space, since this is global analysis after all. In practice rarely seems to be a concern; omake takes up much less space than your web browser even on large projects.

Some notes to the GNU/BSD make user.

3.1  OMakeroot vs. OMakefile

Before we begin with examples, let’s ask the first question, “What is the difference between the project root OMakeroot and OMakefile?” A short answer is, there is no difference, but you must have an OMakeroot file (or Root.om file).

However, the normal style is that OMakeroot is boilerplate and is more-or-less the same for all projects. The OMakefile is where you put all your project-specific stuff.

To get started, you don’t have to do this yourself. In most cases you just perform the following step in your project root directory.

This will create the initial OMakeroot and OMakefile files that you can edit to get started.

3.2  An example C project

To begin, let’s start with a simple example. Let’s say that we have a full directory tree, containing the following files.

    my_project/
    |--> OMakeroot
    |--> OMakefile
    `--> src/
         |---> OMakefile
         |---> lib/
         |     |---> OMakefile
         |     |---> ouch.c
         |     |---> ouch.h
         |     `---> bandaid.c
         `---> main/
               |---> OMakefile
               |---> horsefly.c
               |---> horsefly.h
               `---> main.c

Here is an example listing.

my_project/OMakeroot:
    # Include the standard configuration for C applications
    open build/C
    
    # Process the command-line vars
    DefineCommandVars()
    
    # Include the OMakefile in this directory.
    .SUBDIRS: .

my_project/OMakefile:
    # Set up the standard configuration
    CFLAGS += -g

    # Include the src subdirectory
    .SUBDIRS: src

my_project/src/OMakefile:
    # Add any extra options you like
    CFLAGS += -O2

    # Include the subdirectories
    .SUBDIRS: lib main

my_project/src/lib/OMakefile:
    # Build the library as a static library.
    # This builds libbug.a on Unix/OSX, or libbug.lib on Win32.
    # Note that the source files are listed _without_ suffix.
    StaticCLibrary(libbug, ouch bandaid)

my_project/src/main/OMakefile:
    # Some files include the .h files in ../lib
    INCLUDES += ../lib

    # Indicate which libraries we want to link against.
    LIBS[] +=
        ../lib/libbug

    # Build the program.
    # Builds horsefly.exe on Win32, and horsefly on Unix.
    # The first argument is the name of the executable.
    # The second argument is an array of object files (without suffix)
    # that are part of the program.
    CProgram(horsefly, horsefly main)

    # Build the program by default (in case omake is called
    # without any arguments).  EXE is defined as .exe on Win32,
    # otherwise it is empty.
    .DEFAULT: horsefly$(EXE)

Most of the configuration here is defined in the file build/C.om (which is part of the OMake distribution). This file takes care of a lot of work, including:

Variables are inherited down the hierarchy, so for example, the value of CFLAGS in src/main/OMakefile is “-g -O2”.

3.3  An example OCaml project

Let’s repeat the example, assuming we are using OCaml instead of C. This time, the directory tree looks like this.

    my_project/
    |--> OMakeroot
    |--> OMakefile
    `--> src/
         |---> OMakefile
         |---> lib/
         |     |---> OMakefile
         |     |---> ouch.ml
         |     |---> ouch.mli
         |     `---> bandaid.ml
         `---> main/
               |---> OMakefile
               |---> horsefly.ml
               |---> horsefly.mli
               `---> main.ml

The listing is only a bit different.

my_project/OMakeroot:
    # Include the standard configuration for OCaml applications
    open build/OCaml
    
    # Process the command-line vars
    DefineCommandVars()
    
    # Include the OMakefile in this directory.
    .SUBDIRS: .

my_project/OMakefile:
    # Set up the standard configuration
    OCAMLFLAGS += -Wa

    # Do we want to use the bytecode compiler,
    # or the native-code one?  Let's use both for
    # this example.
    NATIVE_ENABLED = true
    BYTE_ENABLED = true

    # Include the src subdirectory
    .SUBDIRS: src

my_project/src/OMakefile:
    # Include the subdirectories
    .SUBDIRS: lib main

my_project/src/lib/OMakefile:
    # Let's do aggressive inlining on native code
    OCAMLOPTFLAGS += -inline 10

    # Build the library as a static library.
    # This builds libbug.a on Unix/OSX, or libbug.lib on Win32.
    # Note that the source files are listed _without_ suffix.
    OCamlLibrary(libbug, ouch bandaid)

my_project/src/main/OMakefile:
    # These files depend on the interfaces in ../lib
    OCAMLINCLUDES += ../lib

    # Indicate which libraries we want to link against.
    OCAML_LIBS[] +=
        ../lib/libbug

    # Build the program.
    # Builds horsefly.exe on Win32, and horsefly on Unix.
    # The first argument is the name of the executable.
    # The second argument is an array of object files (without suffix)
    # that are part of the program.
    OCamlProgram(horsefly, horsefly main)

    # Build the program by default (in case omake is called
    # without any arguments).  EXE is defined as .exe on Win32,
    # otherwise it is empty.
    .DEFAULT: horsefly$(EXE)

In this case, most of the configuration here is defined in the file build/OCaml.om. In this particular configuration, files in my_project/src/lib are compiled aggressively with the option -inline 10, but files in my_project/src/lib are compiled normally.

3.4  Handling new languages

The previous two examples seem to be easy enough, but they rely on the OMake standard library (the files build/C and build/OCaml) to do all the work. What happens if we want to write a build configuration for a language that is not already supported in the OMake standard library?

For this example, let’s suppose we are adopting a new language. The language uses the standard compile/link model, but is not in the OMake standard library. Specifically, let’s say we have the following setup.

To define a build configuration, we have to do three things.

  1. Define a .SCANNER rule for discovering dependency information for the source files.
  2. Define a generic rule for compiling a .cat file to a .woof file.
  3. Define a rule (as a function) for linking .woof files to produce a .dog executable.

Initially, these definitions will be placed in the project root OMakefile.

3.4.1  Defining a default compilation rule

Let’s start with part 2, defining a generic compilation rule. We’ll define the build rule as an implicit rule. To handle the include path, we’ll define a variable CAT_INCLUDES that specifies the include path. This will be an array of directories. To define the options, we’ll use a lazy variable (Section 7.5). In case there are any other standard flags, we’ll define a CAT_FLAGS variable.

   # Define the catc command, in case we ever want to override it
   CATC = catc

   # The default flags are empty
   CAT_FLAGS =
   
   # The directories in the include path (empty by default)
   INCLUDES[] =

   # Compute the include options from the include path
   PREFIXED_INCLUDES[] = $`(mapprefix -I, $(INCLUDES))

   # The default way to build a .woof file
   %.woof: %.cat
       $(CATC) $(PREFIXED_INCLUDES) $(CAT_FLAGS) -c $<

The final part is the build rule itself, where we call the catc compiler with the include path, and the CAT_FLAGS that have been defined. The $< variable represents the source file.

3.4.2  Defining a rule for linking

For linking, we’ll define another rule describing how to perform linking. Instead of defining an implicit rule, we’ll define a function that describes the linking step. The function will take two arguments; the first is the name of the executable (without suffix), and the second is the files to link (also without suffixes). Here is the code fragment.

    # Optional link options
    CAT_LINK_FLAGS =

    # The function that defines how to build a .dog program
    CatProgram(program, files) =
        # Add the suffixes
        file_names = $(addsuffix .woof, $(files))
        prog_name = $(addsuffix .dog, $(program))

        # The build rule
        $(prog_name): $(file_names)
            $(CATC) $(PREFIXED_INCLUDES) $(CAT_FLAGS) $(CAT_LINK_FLAGS) -o $@ $+

        # Return the program name
        value $(prog_name)

The CAT_LINK_FLAGS variable is defined just in case we want to pass additional flags specific to the link step. Now that this function is defined, whenever we want to define a rule for building a program, we simply call the rule. The previous implicit rule specifies how to compile each source file, and the CatProgram function specifies how to build the executable.

    # Build a rover.dog program from the source
    # files neko.cat and chat.cat.
    # Compile it by default.
    .DEFAULT: $(CatProgram rover, neko chat)

3.4.3  Dependency scanning

That’s it, almost. The part we left out was automated dependency scanning. This is one of the nicer features of OMake, and one that makes build specifications easier to write and more robust. Strictly speaking, it isn’t required, but you definitely want to do it.

The mechanism is to define a .SCANNER rule, which is like a normal rule, but it specifies how to compute dependencies, not the target itself. In this case, we want to define a .SCANNER rule of the following form.

    .SCANNER: %.woof: %.cat
        <commands>

This rule specifies that a .woof file may have additional dependencies that can be extracted from the corresponding .cat file by executing the <commands>. The result of executing the <commands> should be a sequence of dependencies in OMake format, printed to the standard output.

As we mentioned, each .cat file specifies dependencies on .woof files with an open directive. For example, if the neko.cat file contains a line open chat, then neko.woof depends on chat.woof. In this case, the <commands> should print the following line.

    neko.woof: chat.woof

For an analogy that might make this clearer, consider the C programming language, where a .o file is produced by compiling a .c file. If a file foo.c contains a line like #include "fum.h", then foo.c should be recompiled whenever fum.h changes. That is, the file foo.o depends on the file fum.h. In the OMake parlance, this is called an implicit dependency, and the .SCANNER <commands> would print a line like the following.

    foo.o: fum.h

Now, returning to the animal world, to compute the dependencies of neko.woof, we should scan neko.cat, line-by-line, looking for lines of the form open <name>. We could do this by writing a program, but it is easy enough to do it in omake itself. We can use the builtin awk function to scan the source file. One slight complication is that the dependencies depend on the INCLUDE path. We’ll use the find-in-path function to find them. Here we go.

    .SCANNER: %.woof: %.cat
        section
            # Scan the file
            deps[] =
            awk($<)
            case $'^open'
                deps[] += $2
                export

            # Remove duplicates, and find the files in the include path
            deps = $(find-in-path $(INCLUDES), $(set $(deps)))

            # Print the dependencies
            println($"$@: $(deps)")

Let’s look at the parts. First, the entire body is defined in a section because we are computing it internally, not as a sequence of shell commands.

We use the deps variable to collect all the dependencies. The awk function scans the source file ($<) line-by-line. For lines that match the regular expression ^open (meaning that the line begins with the word open), we add the second word on the line to the deps variable. For example, if the input line is open chat, then we would add the chat string to the deps array. All other lines in the source file are ignored.

Next, the $(set $(deps)) expression removes any duplicate values in the deps array (sorting the array alphabetically in the process). The find-in-path function then finds the actual location of each file in the include path.

The final step is print the result as the string $"$@: $(deps)" The quotations are added to flatten the deps array to a simple string.

3.4.4  Pulling it all together

To complete the example, let’s pull it all together into a single project, much like our previous example.

    my_project/
    |--> OMakeroot
    |--> OMakefile
    `--> src/
         |---> OMakefile
         |---> lib/
         |     |---> OMakefile
         |     |---> neko.cat
         |     `---> chat.cat
         `---> main/
               |---> OMakefile
               `---> main.cat

The listing for the entire project is as follows. Here, we also include a function CatLibrary to link several .woof files into a library.

my_project/OMakeroot:
    # Process the command-line vars
    DefineCommandVars()
    
    # Include the OMakefile in this directory.
    .SUBDIRS: .

my_project/OMakefile:
   ########################################################################
   # Standard config for compiling .cat files
   #

   # Define the catc command, in case we ever want to override it
   CATC = catc

   # The default flags are empty
   CAT_FLAGS =
   
   # The directories in the include path (empty by default)
   INCLUDES[] =

   # Compute the include options from the include path
   PREFIXED_INCLUDES[] = $`(mapprefix -I, $(INCLUDES))

   # Dependency scanner for .cat files
   .SCANNER: %.woof: %.cat
        section
            # Scan the file
            deps[] =
            awk($<)
            case $'^open'
                deps[] += $2
                export

            # Remove duplicates, and find the files in the include path
            deps = $(find-in-path $(INCLUDES), $(set $(deps)))

            # Print the dependencies
            println($"$@: $(deps)")

   # The default way to compile a .cat file
   %.woof: %.cat
       $(CATC) $(PREFIXED_INCLUDES) $(CAT_FLAGS) -c $<

   # Optional link options
   CAT_LINK_FLAGS =

   # Build a library for several .woof files
   CatLibrary(lib, files) =
       # Add the suffixes
       file_names = $(addsuffix .woof, $(files))
       lib_name = $(addsuffix .woof, $(lib))

       # The build rule
       $(lib_name): $(file_names)
           $(CATC) $(PREFIXED_INCLUDES) $(CAT_FLAGS) $(CAT_LINK_FLAGS) -a $@ $+

       # Return the program name
       value $(lib_name)

   # The function that defines how to build a .dog program
   CatProgram(program, files) =
       # Add the suffixes
       file_names = $(addsuffix .woof, $(files))
       prog_name = $(addsuffix .dog, $(program))

       # The build rule
       $(prog_name): $(file_names)
           $(CATC) $(PREFIXED_INCLUDES) $(CAT_FLAGS) $(CAT_LINK_FLAGS) -o $@ $+

       # Return the program name
       value $(prog_name)

   ########################################################################
   # Now the program proper
   #

   # Include the src subdirectory
   .SUBDIRS: src

my_project/src/OMakefile:
   .SUBDIRS: lib main

my_project/src/lib/OMakefile:
   CatLibrary(cats, neko chat)

my_project/src/main/OMakefile:
   # Allow includes from the ../lib directory
   INCLUDES[] += ../lib

   # Build the program
   .DEFAULT: $(CatProgram main, main ../cats)

Some notes. The configuration in the project OMakeroot defines the standard configuration, including the dependency scanner, the default rule for compiling source files, and functions for building libraries and programs.

These rules and functions are inherited by subdirectories, so the .SCANNER and build rules are used automatically in each subdirectory, so you don’t need to repeat them.

3.4.5  Finishing up

At this point we are done, but there are a few things we can consider.

First, the rules for building cat programs is defined in the project OMakefile. If you had another cat project somewhere, you would need to copy the OMakeroot (and modify it as needed). Instead of that, you should consider moving the configuration to a shared library directory, in a file like Cat.om. That way, instead of copying the code, you could include the shared copy with an OMake command open Cat. The share directory should be added to your OMAKEPATH environment variable to ensure that omake knows how to find it.

Better yet, if you are happy with your work, consider submitting it as a standard configuration (by sending a request to omake@metaprl.org) so that others can make use of it too.

3.5  Collapsing the hierarchy, .SUBDIRS bodies

Some projects have many subdirectories that all have the same configuration. For instance, suppose you have a project with many subdirectories, each containing a set of images that are to be composed into a web page. Apart from the specific images, the configuration of each file is the same.

To make this more concrete, suppose the project has four subdirectories page1, page2, page3, and page4. Each contains two files image1.jpg and image2.jpg that are part of a web page generated by a program genhtml.

Instead of of defining a OMakefile in each directory, we can define it as a body to the .SUBDIRS command.

    .SUBDIRS: page1 page2 page3 page4
        index.html: image1.jpg image2jpg
            genhtml $+ > $@

The body of the .SUBDIRS is interpreted exactly as if it were the OMakefile, and it can contain any of the normal statements. The body is evaluated in the subdirectory for each of the subdirectories. We can see this if we add a statement that prints the current directory ($(CWD)).

    .SUBDIRS: page1 page2 page3 page4
        println($(absname $(CWD)))
        index.html: image1.jpg image2jpg
            genhtml $+ > $@
  # prints
    /home/jyh/.../page1
    /home/jyh/.../page2
    /home/jyh/.../page3
    /home/jyh/.../page4

3.5.1  Using glob patterns

Of course, this specification is quite rigid. In practice, it is likely that each subdirectory will have a different set of images, and all should be included in the web page. One of the easier solutions is to use one of the directory-listing functions, like glob or ls. The glob function takes a shell pattern, and returns an array of file with matching filenames in the current directory.

    .SUBDIRS: page1 page2 page3 page4
        IMAGES = $(glob *.jpg)
        index.html: $(IMAGES)
            genhtml $+ > $@

3.5.2  Simplified sub-configurations

Another option is to add a configuration file in each of the subdirectories that defines directory-specific information. For this example, we might define a file BuildInfo.om in each of the subdirectories that defines a list of images in that directory. The .SUBDIRS line is similar, but we include the BuildInfo file.

    .SUBDIRS: page1 page2 page3 page4
        include BuildInfo   # Defines the IMAGES variable

        index.html: $(IMAGES)
            genhtml $+ > $@

Where we might have the following configurations.

   page1/BuildInfo.om:
       IMAGES[] = image.jpg
   page2/BuildInfo.om:
       IMAGES[] = ../common/header.jpg winlogo.jpg
   page3/BuildInfo.om:
       IMAGES[] = ../common/header.jpg unixlogo.jpg daemon.jpg
   page4/BuildInfo.om:
       IMAGES[] = fee.jpg fi.jpg foo.jpg fum.jpg

3.5.3  Computing the subdirectory list

The other hardcoded specification is the list of subdirectories page1, ..., page4. Rather than editing the project OMakefile each time a directory is added, we could compute it (again with glob).

    .SUBDIRS: $(glob page*)
        index.html: $(glob *.jpg)
            genhtml $+ > $@

Alternately, the directory structure may be hierarchical. Instead of using glob, we could use the subdirs function, returns each of the directories in a hierarchy. For example, this is the result of evaluating the subdirs function in the omake project root. The P option, passed as the first argument, specifies that the listing is “proper,” it should not include the omake directory itself.

    osh> subdirs(P, .)
    - : <array
            /home/jyh/.../omake/mk : Dir
            /home/jyh/.../omake/RPM : Dir
            ...
            /home/jyh/.../omake/osx_resources : Dir>

Using subdirs, our example is now as follows.

    .SUBDIRS: $(subdirs P, .)
        index.html: $(glob *.jpg)
            genhtml $+ > $@

In this case, every subdirectory will be included in the project.

If we are using the BuildInfo.om option. Instead of including every subdirectory, we could include only those that contain a BuildInfo.om file. For this purpose, we can use the find function, which traverses the directory hierarchy looking for files that match a test expression. In our case, we want to search for files with the name BuildInfo.om. Here is an example call.

    osh> FILES = $(find . -name BuildInfo.om)
    - : <array
            /home/jyh/.../omake/doc/html/BuildInfo.om : File
            /home/jyh/.../omake/src/BuildInfo.om : File
            /home/jyh/.../omake/tests/simple/BuildInfo.om : File>
    osh> DIRS = $(dirof $(FILES))
    - : <array
            /home/jyh/.../omake/doc/html : Dir
            /home/jyh/.../omake/src : Dir
            /home/jyh/.../omake/tests/simple : Dir>

In this example, there are three BuildInfo.om files, in the doc/html, src, and tests/simple directories. The dirof function returns the directories for each of the files.

Returning to our original example, we modify it as follows.

    .SUBDIRS: $(dirof $(find . -name BuildInfo.om))
        include BuildInfo   # Defines the IMAGES variable

        index.html: $(IMAGES)
            genhtml $+ > $@

3.5.4  Temporary directories

Sometimes, your project may include temporary directories–directories where you place intermediate results. these directories are deleted whenever the project is cleanup up. This means, in particular, that you can’t place an OMakefile in a temporary directory, because it will be removed when the directory is removed.

Instead, if you need to define a configuration for any of these directories, you will need to define it using a .SUBDIRS body.

    section
        CREATE_SUBDIRS = true

        .SUBDIRS: tmp
            # Compute an MD5 digest
            %.digest: %.comments
               echo $(digest $<) > $@

            # Extract comments from the source files
            %.comments: ../src/%.src
               grep '^#' $< > $@

            .DEFAULT: foo.digest

    .PHONY: clean

    clean:
        rm -rf tmp        

In this example, we define the CREATE_SUBDIRS variable as true, so that the tmp directory will be created if it does not exist. The .SUBDIRS body in this example is a bit contrived, but it illustrates the kind of specification you might expect. The clean phony-target indicates that the tmp directory should be removed when the project is cleaned up.

Jump to:  OMake Home • Guide Home • Guide (single-page) • Contents (short) • Contents (long)
Index:  All • Variables • Functions • Objects • Targets • Options