Hi Malcolm,
>>>>> "Malcolm" == Malcolm Cohen <[log in to unmask]> writes:
Malcolm> Ted Stern said:
>> The issue of chain recompilation has been hashed around in this group
>> for years. Many solutions have been offered but as John Bray has
>> pointed out, no really satisfactory solution has come forward. You
>> either have a bunch of "touch" files scattered around or you have to
>> call MAKE twice to get everything updated.
I'm attaching updated versions of my examples. I should clarify that I am
thinking about an extreme case --- you might have a huge directory tree of
.f90 files and want to build in a similar tree of object files. I routinely
deal with builds of around 8K f77 sources, and I want to address how one would
handle this sort of situation when f90 module dependencies are considered.
Malcolm> Despite believing this myself for a long time, a little while ago
Malcolm> someone pointed out to me a simple scheme which avoids these
Malcolm> problems. An example of how to do it is attached.
Malcolm> But first, the explanation:
Malcolm> (1) Just do the Makefile as per usual, with the .o files
Malcolm> depending on the USEd .mod files, however the rules for making
Malcolm> the .mod files should not have any dependencies - instead, they
Malcolm> just (delete and then) make the relevant .o files. This is the
Malcolm> crucial point - if the .mod files have dependencies they will be
Malcolm> regenerated unnecessarily.
This means that a particular object file could result from more than one
target rule. But MAKE would not know that. You would be lying to MAKE. This
is a Bad Thing and will eventually get you into trouble ;-).
Malcolm> (2) Files which contain modules should use a script
Malcolm> e.g. "checkf95" which does a comparison of the .mod files.
I suggested using "cmp -s". Essentially the same thing. See
http://www.theochem.uwa.edu.au/fortran/recompile/
for a discussion of this point.
For my proposed method, there are several simplifications possible: if you
discipline the programmer and have a compiler that ensures that
- there is only one module per file,
- the module name has exactly the same stem as the object file,
- you are compiling in the object directory so no paths are required,
you can use $*.o in your compilation rules to designate the resulting object
file. The main new thing I've contributed is a special GNU Make function to
either do the compilation or touch the results, using an implicit pattern rule:
compile_f90 = $(FC) -c $(FFLAGS) $< -o $*.o
%.o %.mod: %.f90
@$(call INCR_BUILD,$(compile_f90),$*.o,$(objmods),%.mod)
My perl script can be simplified dramatically to generate output like
filename.o \
filename.mod: objmods := filename.mod
filename.o: another.mod andanother.mod
But regarding those coding guidelines: in general, how does a .mod MAKE target
rule know what .o file to generate? What if the resulting .mod file has a
completely different name than the associated object file or source file?
What if the source file has mixed lower and upper case? While you can direct
object output to a file with the same stem as the source, generated modules
are usually lowercased or uppercased versions of the module name. And what if
there is more than one module per file?
It is possible that for one or more compilers or platforms, you may not be
able to enforce the requirements.
Malcolm> No special tools are needed, though the checkf95 script I've
Malcolm> provided exploits the NAG compiler's ability to suppress .mod
Malcolm> file output or to only produce .mod file output.
I was thinking of compiler/platform independence ;-).
Malcolm> Nor have I supplied a clever perl script to generate Makefiles
Malcolm> like this. I'm sure someone else could throw one together
Malcolm> though...
The perl script I provided doesn't generate Makefiles --- the Makefile
generates the Makefile ;-). The perl script generates a dependency list and
the target rule for a particular object and its side-effect modules. In the
special case I describe above, an explicit target rule is not necessary --
only the dependency list is required, and a special target-specific variable
to reference the resulting module file.
Malcolm> Aside: There is one slight deficiency of the current NAG f95 with
Malcolm> this scheme, which I'll explain with reference to the example: if
Malcolm> a.f90 is changed in a way that does not affect a.mod, and then
Malcolm> b.f90 is changed, all subsequent compilations of files that USE B
Malcolm> will complain about a.mod being out of date (it's just a warning,
Malcolm> but irritating nonetheless).
That was precisely the problem I was addressing. I handle it in such a way
that MAKE knows exactly what is going on.
Malcolm> We've added an option "-nocheck_modtime" to suppress this
Malcolm> message - the message is really only there for people who are not
Malcolm> using "make" or whose Makefiles are broken.
IMO, the Makefile is "broken" if your dependencies are perpetually out of date
;-). The technique I proposed (based on contibutions of *many* others)
ensures that your dependency graph is always complete and always updated.
For many people with simple projects, a simple technique is adequate, I agree.
For them, a simplified version of my mkdepf90.perl script would be sufficient,
and I can provide one if you like.
Ted
--
============================================================
Ted Stern Engineering Applications
Cray Inc. http://www.cray.com
411 First Avenue South, Suite 600 206-701-2182
Seattle, WA 98104-2860 FAX: 206-701-2500
============================================================
#
# Some utility functions used to handle incremental recompilations:
#
# Single argument function that echoes argument before running it:
ECHO_AND_RUN = \
/bin/echo "\n$(1)\n" ; $(1);
#
# Single argument function to create "old" files for dependency checks:
# $1 = list of results
#
MAKE_BACKUPS = \
for file in $(1) ; do \
test -s $${file} && \
cp -p $${file} $${file}~ || \
touch $${file}~ ; \
done;
#
# 3 argument incremental recompile utility function that does comparison.
# If the old files differ, the compile command is run, otherwise
# the results are touched only.
# $1 = compile command
# $2 = list of dependencies
# $3 = list of results
#
TOUCH_OR_BUILD = \
i=0 ; \
for file in $(2) ; do \
cmp -s $${file} $${file}~ || i=1 ; \
done ; \
if test $${i} -eq 1 ; then \
$(call ECHO_AND_RUN,$(1)) \
else \
$(call ECHO_AND_RUN,touch $(3)) \
fi;
# INCR_BUILD:
# Four argument general purpose incremental build function
# $1 = build command
# $2 = main build target
# $3 = side results of build
# $4 = pattern to look for in dependencies
#
# The first $(if ...) generates backup side-results files
# The second $(if ...) checks whether there are auto-generated dependencies
# that are out of date. If not, a regular compilation is done.
# If there are out-of-date auto-generated dependencies, and there are
# other dependencies, normal compilation is done, otherwise the special
# TOUCH_OR_BUILD utility is called.
#
define INCR_BUILD
$(if $(3),\
$(call MAKE_BACKUPS,$(3)))\
$(if $(filter $(4),$?),\
$(if $(filter-out $(4),$?),\
$(call ECHO_AND_RUN,$(1)),\
$(call TOUCH_OR_BUILD,$(1),$(filter $(4),$?),$(2) $(3))),\
$(call ECHO_AND_RUN,$(1)))
endef
#EOF
#!/usr/bin/perl
#
# $Id: mkdepf90.perl,v 1.13 2001/12/07 20:08:14 stern Exp $
# Author: Ted Stern <[log in to unmask]>
#
# Usage:
#
# mkdepf90.perl -src srcfile [-objdir objdir]
#
# Produces two files as output. If `basename srcfile .f90` is srcbase,
# the output goes to two files
#
# objdir/srcbase.rule and objdir/srcbase.deps
#
# - Intended for use with IncrBuild.mk in GNU Make versions 3.79.1 and higher.
#
# - Generates macros, target rules, and dependency listings for f90 files
# that USE or define modules.
#
# - Intended for one source file at a time. For example, use in a Makefile
# with the following pattern rule and include statement:
#
# $(OBJDIR)/%.rule $(OBJDIR)/%.deps: $(SRCDIR)/%.f90
# mkdepf90.perl -src $< -objdir $(@D)
#
# # include variable definitions and target rules:
# include $(F90_objs:.o=.rule)
#
# # include extra dependencies:
# include $(F90_objs:.o=.deps)
#
# - It doesn't matter if the specified object directory has a trailing
# slash or not.
#
# Acknowledgements:
# Cribbed basic dependency text-processing logic from makemake.perl[1],
# with some argument parsing from fmkmf[4].
#
# References:
# [1] http://www.fortran.com/makemake.perl
# [2] http://www.theochem.uwa.edu.au/fortran/recompile/
# [3] http://www.helsinki.fi/~eedelman/makedepf90.html
# [4] http://www.met.ed.ac.uk/~hcp/fmkmf.html
# [5] http://www.gfdl.gov/~vb/mkmf.html
# [6] http://www.canb.auug.org.au/~millerp/rmch/recu-make-cons-harm.html
# [7] http://www.paulandlesley.org/gmake/
#
#-----------------------------------------------------------------------
# Implementation comments:
#
# *** NOTE ***
# Modify the &ModCase function for your particular compiler!
# It is currently set up for Cray UNICOS compilers, which convert
# module names into uppercase; e.g., "module a_def" causes the creation
# of A_DEF.mod when the -em compile option is specified. On many other
# systems, module files are created with lowercased names.
# *** NOTE ***
#
# Target-specific variables are defined so that both object and module
# targets can compile to the same target, $(objpath) (a MAKE variable
# reference, not a perl variable!).
#
# Example of simple output for no OBJDIR and no modules created:
#
# mkdepf90.perl -src filename.f90
#
# output in filename.rule:
#
# # Variable containing the full path to the object:
# filename.o := filename.o
#
# # Target-specific variable:
# # objpath = the full path to the f90 obj file:
# $(filename.o): objpath := $(filename.o)
#
# # Target-specific variable:
# # objbase = the f90 obj file stripped of directory component:
# $(filename.o): objbase := filename.o
#
# # The main target rule:
# $(filename.o): filename.f90
# $(COMPILE_F90)
#
# output in filename.deps:
#
# # Extra dependencies of the obj file:
# $(filename.o): $(A_DEF.mod) file1.h file2.inc
#
# Example of complex case with -objdir specified, multiple modules created,
# and multiple module and include dependencies:
#
# mkdepf90.perl -src complex_def.f90 -objdir $(OBJDIR)/indep/c/
#
# output in $(OBJDIR)/indep/c/complex_def.rule:
#
# # Using objdir $(OBJDIR)/indep/c/ from cmd line
# # Variable containing the full path to the object:
# complex_def.o := $(OBJDIR)/indep/c/complex_def.o
#
# # For each defined module, define the dir-stripped module name
# # as the full path to the generated module:
# A_DEF.mod := $(OBJDIR)/indep/c/A_DEF.mod
#
# B_DEF.mod := $(OBJDIR)/indep/c/B_DEF.mod
#
# # Variable containing the complete list of generated modules:
# complex_def.o.mods := $(A_DEF.mod) $(B_DEF.mod)
#
# # Target-specific variable:
# # objmods = the modules associated with the target list:
# $(complex_def.o) \
# $(complex_def.o.mods): objmods := $(complex_def.o.mods)
#
# # Target-specific variable:
# # objpath = the full path to the f90 obj file:
# $(complex_def.o) \
# $(complex_def.o.mods): objpath := $(complex_def.o)
#
# # Target-specific variable:
# # objbase = the f90 obj file stripped of directory component:
# $(complex_def.o) \
# $(complex_def.o.mods): objbase := complex_def.o
#
# # The main target rule:
# $(complex_def.o) \
# $(complex_def.o.mods): complex_def.f90
# $(COMPILE_F90)
#
# output in $(OBJDIR)/indep/c/complex_def.deps:
#
# # Extra dependencies of the obj file:
# $(complex_def.o): $(C_DEF.mod) $(D_DEF.mod) $(E_DEF.mod) \
# $(F_DEF.mod) file1.h file2.inc file3.inc
#
#-----------------------------------------------------------------------
# Here's what does the work (argument parsing
while (@ARGV){
$arg=shift;
if ($arg =~ /^-src$/){
$filename=shift;
}
if ($arg =~ /^-objdir$/){
$objdir=shift;
}
}
if ( not $filename ) {
die(
"Expecting 1 to 2 arguments\n",
"Usage:\n",
"\t\tmkdepf90.perl -src srcfile [-objdir objdir]\n",
"srcfile is required, objdir is optional. objdir may have\n",
"a trailing slash -- it doesn't matter. If objdir is not\n",
"entered, the current working directory is used for output.\n",
"Output:\n",
"\tobjdir/srcbase.rule \& objdir/srcbase.deps\n");
}
if ( $objdir ) {
$objdir =~ s/\/*$//; # remove trailing slashes
$objdir =~ s/\.//; # ignore "." directory
# $prefix stores the object directory without trailing slashes
if ( $objdir ) {
($prefix=$objdir) =~ s/$/\//;
}
}
# Get the base part of the filename (remove everything after last .):
($base = $filename) =~ s/\.[^\.]+$//;
# Strip everything up to last slash:
$base =~ s/^.*\///;
# Prepend the object path onto the objfile name, add .o extension
($objfile = $base) =~ s/.*/$prefix$&.o/;
# Make the *.rule and *.deps filenames:
($rulefile = $base) =~ s/.*/$prefix$&.rule/;
($depsfile = $base) =~ s/.*/$prefix$&.deps/;
# This variable contains the directory-stripped object file name with
# a .o extension:
($objbase = $base) =~ s/$/.o/;
undef @incs;
undef @modules;
undef @dependencies;
undef @moddefs;
undef @modlist;
open(FILE, $filename) or die "Cannot open $filename: $!\n";
unlink $rulefile;
open(OUTFILE,">" . $rulefile);
# Search for includes and USE calls:
while (<FILE>) {
/^\s*(\#|\?\?)*\s*include\s+[\"\']([^\"\']+)[\"\']/i && push(@incs, $2);
/^\s*use\s+([^\s,!]+)/i && push(@modules, &ModCase($1));
/^\s*module\s+([^\s,!]+)/i && push(@moddefs, &ModCase($1));
}
# Define the MAKE macro $($objbase) as the full path to
# the object. E.g., 'foo.o = /path/to/foo.o'
print OUTFILE "# Variable containing the full path to the object:\n";
print OUTFILE "$objbase := $objfile\n\n";
$targlist = "\$($objbase)";
# If modules are defined, create macros for modules and
# a combined object+module target rule:
if (defined @moddefs) {
# @modlist stores module names with .mod on the end
foreach $module (&uniq(sort(@moddefs))) {
($name = $module) =~ s/$/.mod/;
push(@modlist, $name);
}
# Redefine @moddefs to the new @modlist
@moddefs = @modlist;
# Redfine @modlist with each element of @moddefs surrounded by
# MAKE parens:
undef @modlist;
foreach $module (@moddefs) {
($name = $module) =~ s/.*/\$\($&\)/;
push(@modlist, $name);
}
print OUTFILE "# For each defined module, define the dir-stripped module name\n";
print OUTFILE "# as the full path to the generated module:\n";
foreach $module (@moddefs) {
# Define the MAKE macro $(modnamebase.mod) as the full path to the
# module
print OUTFILE "$module := $prefix$module\n\n";
}
# This defines the macro $(objbase.o.mods) as the list of
# modules dependent on the object.
print OUTFILE "# Variable containing the complete list of generated modules:\n";
print OUTFILE "$objbase.mods := ";
$j = length($objbase) + 9;
&PrintWords($j, 0, @modlist);
print OUTFILE "\n\n";
$targlist = "\$($objbase) \\\n\$($objbase.mods)";
print OUTFILE "# Target-specific variable:\n";
print OUTFILE "# objmods = modules associated with the target list:\n";
print OUTFILE "$targlist: objmods := \$($objbase.mods)\n\n";
}
print OUTFILE "# Target-specific variable:\n";
print OUTFILE "# objpath = full path to the f90 obj file:\n";
print OUTFILE "$targlist: objpath := \$($objbase)\n\n";
print OUTFILE "# Target-specific variable:\n";
print OUTFILE "# objbase = f90 obj file stripped of directory component:\n";
print OUTFILE "$targlist: objbase := $objbase\n\n";
print OUTFILE "# The main target rule:\n";
print OUTFILE "$targlist: $filename\n\t\$(COMPILE_F90)\n\n";
# Stop writing to .rule file, start writing to .deps file:
close(OUTFILE);
unlink $depsfile;
open(OUTFILE,">" . $depsfile);
# Print out extra dependencies for the object file beyond the default:
if (defined @incs || defined @modules) {
foreach $module (&uniq(sort(@modules))) {
($name = $module) =~ s/.*/\$\($&.mod\)/;
push(@dependencies, $name);
}
print OUTFILE "# Extra dependencies of the obj file:\n";
print OUTFILE "\$($objbase): ";
$j = length($objbase) + 5;
&PrintWords($j, 0, @dependencies, &uniq(sort(@incs)));
}
print OUTFILE "\n"; # Add newline to .deps file
close(OUTFILE);
# End of processing. Below are utility routines:
#-----------------------------------------------------------------------
# Internal text formatting functions, copied from makemake.perl.
# Note that &ModCase should be modified to handle the module-name
# generation format of your compiler.
#-----------------------------------------------------------------------
# &PrintWords(current output column, extra tab?, word list); --- print words
# nicely
#
sub PrintWords {
local($columns) = 78 - shift(@_);
local($extratab) = shift(@_);
local($wordlength);
#
print OUTFILE "$_[0]";
$columns -= length(shift(@_));
foreach $word (@_) {
$wordlength = length($word);
if ($wordlength + 1 < $columns) {
print OUTFILE " $word";
$columns -= $wordlength + 1;
}
else {
#
# Continue onto a new line
#
if ($extratab) {
print OUTFILE " \\\n\t\t$word";
$columns = 62 - $wordlength;
}
else {
print OUTFILE " \\\n\t$word";
$columns = 70 - $wordlength;
}
}
}
}
#-----------------------------------------------------------------------
# &ModCase(string); --- convert string into correct case for module names
#
sub ModCase {
local($string) = $_[0];
# Lowercase on non-UNICOS:
# $string =~ tr/A-Z/a-z/;
## Uppercase on Cray UNICOS
$string =~ tr/a-z/A-Z/;
$string;
}
#-----------------------------------------------------------------------
# &uniq(sorted word list); --- remove adjacent duplicate words
#
sub uniq {
local(@words);
foreach $word (@_) {
if ($word ne $words[$#words]) {
push(@words, $word);
}
}
@words;
}
|
