The M command does not actually extract the temporary files and run DIFF3 with them. Instead, the M command creates a file named MIGRATE2.BAT, which contains TLIB, DIFF3, and SCOPY commands to do the actual migration.

This gives you the opportunity to examine what TLIB will be doing, and intervene, if you wish.

For instance, you may need to run a “pretty-printer” (source code reformatter) on the temporary files before doing the DIFF3 merge. Such a step is not recommended unless one of the versions has been reformatted already, in which case DIFF3 would otherwise be unable to do a proper merge because of the pervasiveness of the changes.

Generally, you should first run MIGRATE2.BAT to do the merges, and then examine any modules for which it warns that there were “collisions” (conflicts) between the merged versions. The collisions are marked with distinctive “flag lines.” (The flag lines normally containing “###”, which you can search for with your editor, but they can be changed with the D3COLLIDE configuration parameter).

If any of the merged files have been reformatted, it will be all too obvious when you look at the huge “collision areas” marked by DIFF3. For such cases, you can edit the MIGRATE2.BAT file to make it reformat the three input files before running DIFF3, to make them more consistent with one another (use the same pretty-printer that caused the problem, if possible). Then you can run the modified MIGRATE2.BAT again to retry the merge.

Other “collisions,” if there were any, must be resolved manually, by examining the flagged collision areas and correcting the problems with a text editor. Be sure to leave intact the special “[MERGED_...]” and “[COPIED_...]” comments at the beginning of the files.

Hint: To capture a “log” of everything that happened when you ran MIGRATE2.BAT, use Chris Dunford's wonderful “freeware” CONCOPY utility. With Mr. Dunford's kind permission, we've included a copy of CONCOPY with the shareware and public domain utilities collection on the TLIB CD.

After you've used the M command to migrate your changes, and you've run MIGRATE2.BAT to do any DIFF3 merges that were needed, you can delete MIGRATE2.BAT.

Then you should test your new versions. In our experience, if there were no collisions, the merged version almost always works correctly. However, this is not guaranteed. In fact, it seems rather surprising, since there are many ways for two different changes to be incompatible without having obvious textual collisions that are detectable by DIFF3.

For example, the two merged versions could each have added a new variable, for completely unrelated reasons, but with the same name.

Nevertheless, such cases don't seem to crop up very often.

After you have finished testing, you can use the TLIB U (update) command to store the new versions of your source files in their TLIB libraries, and the TLIB S (snapshot) command to label the new versions.

MSS and MFSS -- Overriding the “Base” Version

Occasionally you may want to override TLIB's determination of the most recent common ancestor of the two versions to be merged. For this, TLIB provides the MSS and MFSS commands.

The MSS (and MFSS) commands take four parameters:

TLIB MSS filename.ext toVer baseVer fromVer

For example, suppose that version 7 of myfile.prg is the latest "standard" version, and version 7.(5)3 is a customized variant of it. Now, suppose that you fix a bug in version 7.(5)3, and store it as version 7.(5)4, but the bug was also present in the standard version.

Of course, the problem would have been simpler if you'd put the bug fix onto the standard version first, instead of the customized version. Then you could have simply migrated all improvements to date into from the standard version into the customized version. Unfortunately, sometimes you may not have the luxury of making that choice; if the customer has a "sev 1" problem, you may have no choice but to first implement the fix in his customized version.

In that case you cannot use the M or MS command to merge the bug fix into version 7, because TLIB has no way of knowing that only the last revision to this customized variant is applicable to the standard version. If you were to do TLIB MS myfile.prg * 7.(5)4 then TLIB would add all the changes between versions 7 and 7.(5)4 to the latest trunk version, with the result that the customizations would also become part of the standard version (which is not what you want).

Instead, you should use the MSS command, to tell TLIB that you want only the changes between versions 7.(5)3 and 7.(5)4 added to the standard version:

TLIB MSS myfile.prg * 7.(5)3 7.(5)4

The MFSS command is just like the MSS command, except that TLIB will silently skip files for which no action was needed. That doesn't do any good for the example above, but is can be useful when the changes must be made to many files.

Suppose, for example, that lastweek.snp is a snapshot taken (with the S command) of your customized project level before you made an elaborate set of changes to many of the source files in the customized level, to fix a complex bug. Also, assume that the standard level's project level tracking file is f:\std\tlibwork.trk and the customized level's project level tracking file is f:\cus\tlibwork.trk.

You need to migrate that bug fix into the standard project level. To do so, you would do the following command (while working at the standard level):

TLIB MFSS *.* @f:\std\tlibwork.trk @lastweek.snp @f:\cus\tlibwork.trk
(should be all on one line)

Or, if the standard version is always the latest trunk versions, this does the same thing:

TLIB MFSS *.* * @lastweek.snp @f:\cus\tlibwork.trk

That tells TLIB to migrate into the current (standard) level all changes made to the customization level since the lastweek.snp snapshot was taken.

As with all of the Migrate commands, the result is a batch script, MIGRATE2.BAT, which you must run to finish the migration. After you have done so, you will be left with a set of checked-out files which have had the needed changes made to them. If there were any "conflicts" which TLIB's DIFF3 utility was unable to resolve (i.e., if your fix changed lines that were specific to the customized version), then you'll have to resolve the conflicts manually. They will be flagged in the source files with distinctive text lines, “###Change collision detected” (or whatever you have configured for the D3COLLIDE configuration parameter), so that you can easily find them with TLIBSCAN or your favorite GREP utility.

A comparison between the M, MS, and MSS commands may be instructive. They all do the same thing, except that with the MSS command you must specify all three version numbers for the merge, but the MS and M commands determine one or two of the versions for you automatically.

With the MSS command, you specify a total of three versions, and what is to be merged into the first of them is specified by a pair of versions (baseVer and fromVer):

TLIB MSS filename.ext toVer baseVer fromVer

A Limitation

The M command does not handle added, deleted, excluded or renamed files. You'll have to handle such files manually.

A Note About Performance

The current TLIB 5.xx release may seem unpleasantly sluggish when you migrate changes from a snapshot or tracking file like this:

TLIB M *.* @snapfile.ext

This simply makes a file list with the names in snapfile.ext, and performs the M command with just those files (instead of “*.*”).

The special TLIB “-n” and “-t” command-line options

TLIB supports two command-line options which are intended mainly for use by the M (migrate) command in the generated MIGRATE2.BAT file. In command-line versions of TLIB, they are supported on the command line only, not in interactive mode. However, they serve to make MIGRATE2.BAT more concise than it would otherwise be.

a) The -t (temporary file) option. -t is just a shorthand equivalent for:

C "locking_n" C "track_n" C "readonlyb_n"

and:

C "keyflag" C "logflag"

To create a journal file, you must specify the file name using JFILE. For instance, if you want the journal to be named JOURNAL.DAT in the root directory of the C: drive, you could specify:

JFILE C:\JOURNAL.DAT

The JOPTIONS configuration parameter is optional. It allows you to select the kinds of information which you wish to have stored in the journal file. There are currently seven legal option characters:

U - commands which update the library

I - commands which check-in a file (with locking enabled)

O - commands which check-out a file (with locking enabled)

B - commands which extract a file, even in browse mode

C - the 1st comment line

A - all comment lines

P - commands which alter a project level

The default configuration value for JOPTIONS is:

JOPTIONS UOCAP

This causes TLIB to record in the journal all commands which update the library (since U is specified) or modify a project level (since P is specified), and all check-out (locking) operations (since O is specified). Additionally, the comment lines associated with every update operation will be recorded (since C and A are specified).

To record only the first comment line (instead of all comments), you can remove the “A” option letter, configuring like this:

JOPTIONS UOCP

Unless A is specified, every journal entry is one line, consisting of 13 fields separated by 12 commas. The last field is the (optional) comment line, which is surrounded by quote marks and is the only field that can contain blanks. When additional comment lines are recorded in the journal (because the A option is set), the first 12 fields are null (that is, the line begins with 12 consecutive commas).

This “comma-delimited” format is consistent with the input requirements of common data base programs, which can be used for monitoring and report generation from the journal file by project managers.

We've also included a sample AWK program, JOURNAL.AWK, to parse the journal file.

Note: Your report generator should not read the journal file directly, since while the journal file is open TLIB will be prevented from accessing it. Instead of reading the journal file directly, use the included SCOPY (shared access file copy) program, SCOPY.EXE, to make a copy of the journal file, and then run your report generator against the copy. For usage instructions, run SCOPY with no parameters.

The 13 fields in each journal line are:

1	date of operation (in TLIB format, DD-MMM-YY, e.g. 19-Oct-97)
2	time of operation (in TLIB format, HH:MM:SS)
3	command (e.g., UF for fast-update)
4	locking mode: N for locking disabled W or Y for whole libraries locked B for branch/level locking weak for weak locking
5	current user ID
6	library file path
7	text file path
8	lock file path (if command did check-in or check-out)
9	version number
10	source file's “key” (name) in the tracking files
11	current project level
12	other project level used, if any (see below)
13	quoted comment line (1st comment line includes source file date/time)

Examples (note the subtle difference between them):

whereis cd:*.c,*.h (find all *.c and *.h files on C: and D:)
whereis cd:*.c *.h (find all *.c files on C: and D:, and all
*.h files on the current drive)

WHEREIS exits with errorlevel 1 if no matching files were found; otherwise it exits with errorlevel 0.

WHEREIS supports many command-line options; run it with no parameters for detailed instructions.

Quiet mode

TLIB 5.0 added support for the -Q (“quiet”) command line option, to suppress display of the TLIB “banner line” (with the copyright notice, etc.) as well as a few other “noise” messages which TLIB normally displays.

The -Q (or -q, or -q1) option, if used, should be the first thing on the TLIB command line.

You can also specify “-q2”, which is just like the “-q” or “-q1” option except that it also suppresses display of any user-configured BANNER lines (see the BANNER and NUMBANNER configuration parameters, p. ).

Actually, TLIB is not really very quiet even with the -Q option. The -Q option only makes TLIB a bit less verbose.

See also the QUIET configuration parameter, p. , which is similar but does not suppress the TLIB copyright banner.

Redirecting standard output

An unusual feature of TLIB (command-line versions) is that it will echo error/warning messages and prompts to “standard error” (stderr) as well as “standard output” (stdout) when stdout has been redirected.

POKETRAK is a little utility for “poking” (changing) specific fields and records in a TLIB version tracking file.

Run it with no parameters for help.

COPYTRAK is a little utility for sorting, copying and otherwise modifying entire TLIB version tracking files.

Run it with no parameters for help.

Note that you should not replace a TLIBWORK.TRK file with a sorted or otherwise reordered one if anyone might be running a copy of TLIB at the time and accessing that file, since TLIB “remembers” the locations of the records in the tracking files, and does not expect them to move.

Saving Disk Space

When you are not working on a particular customized version, you can save disk space by deleting the source code. First, of course, you'll want to make sure that your TLIB libraries are fully up to date, and that the tracking file, TLIBWORK.TRK, contains the correct version information for each module.

A simple way to do both things in one step is to utilize TLIB's DELETESRC configuration parameter, like this:

TLIB C "DELETESRC Y" U *.C

Or, you could update all the libraries with the U command and then just “DEL *.C” or even “DEL *.*”. That sounds radical, but the TLIBWORK.TRK file is stored as a read-only file to prevent it from being deleted. Be careful, though, lest you accidentally delete something else of value (e.g., TLIB.CFG).

Note: the best time to do this is right after you have backed-up your hard disk. Otherwise, you are “putting all your eggs in one basket,” and if a hard disk failure damages one of your TLIB libraries, you may lose your only copy of that module.

Note: you can use the DOS “ATTRIB” command, or TLIB's WHEREIS.EXE program (p. 10), to list the files and to tell which one(s) have the read-only and/or archive attributes set. For instance, “whereis .\tlibwork.trk” will display the attributes of the TLIB version tracking file.

Temporary Files

Note: this section discusses the temporary files that TLIB creates and uses. If you want to know how to extract temporary copies of source files from TLIB see pp. and .

TLIB 5.50 sometimes needs to create one or more (usually small) temporary files, usually called $TLIB_TM.0 or $TLIB_TM.2 (but any name from $TLIB_TM.0 through $TLIB_TM.9 is possible). TLIB will create these files in one of four possible places, in the following order of preference:

o Where your TMP environment variable, if any, indicates. For example, you could put “SET TMP=D:\” in your autoexec.bat file (or in config.sys under OS/2) to make TLIB put its temporary files on the D: drive.

o Where your TEMP environment variable, if any, indicates.

o In the current directory.

o If all else fails, as a last resort TLIB tries to create its temporary files in “C:\”.

Note that TMP (or TEMP) must be a real environment variable; TLIB will not use TMP or TEMP settings from your TLIB.CFG or AUTOSET.BAT file.

Performance will be enhanced slightly if TMP (or TEMP) is set to a ramdisk (VDISK).

Network Bug Workarounds

TLIB has code to detect (and avoid the worst consequences of) certain potentially-catastrophic network bugs.

Some network software caches writes and immediately returns a “success” result after any attempt to write a file, before the file has actually been successfully written. If the write then fails for some reason (e.g., because of a hardware problem, a disk-full condition, or a per-user disk space allocation limit), the program writing the file may not see any indication of the failure.

That can be a disaster! If the file being written was a TLIB library, it is critically important that it be written correctly. For instance, if DELETESRC Y is configured, TLIB deletes the source file after finishing an update, so if the library wasn't correctly written your source code could be lost!

So, we included code in TLIB to check the size of the updated library file after it is closed, to make very sure that the update was successful. If the library file is too small, TLIB now aborts with an error message, without deleting the source file (even if DELETESRC Y is configured).

This almost always prevents any data loss. The one exception that we know of is a network which uses Windows-NT 3.5 and the NTFS file system on the server. That version of Windows-NT 3.5 contains a cache-coherency bug that can occasionally substitute zeros for data on the server, even though the program that wrote the data can read it back correctly. It is not possible for TLIB (or any other program) to detect and work around this bug. Fortunately, Microsoft fixed this bug with Service Pack 2 for NT 3.5. So, never use a Windows-NT 3.5 server with NTFS unless you've applied service pack 2!

The most common network-related problem for TLIB users has to do with file ownership issues under Novell NetWare. Furtunately, this problem is not a threat to data integrity. The problem results from the Novell facility for limiting disk usage by any one user. If you're using this facility, then you should be aware that when someone adds another version to a TLIB library, the storage used accrues to the "owner" of the TLIB library (usually the person who first created the library with the N command), regardless of who is doing the TLIB U command. Thus, you can "run out" of disk space because someone else has used up their allotment.

To determine who owns a TLIB library, use the NDIR command. To determine whether the owner has used up his disk allotment, log in under his user ID and use the CHKVOL command.

If the owner user ID is no longer defined (say, because the employee left the company) then that ID has no disk allotment, and the errors can occur regardless of how much space they were once allotted. If this is what caused the problem, then you need to use FILER to change the NetWare owner ID for each of the TLIB library (or journal or tracking) files.

The READ_ME.TOO file that comes with TLIB has up-to-date information on using TLIB with many different kinds of networks.

Listbld – file list builder

LISTBLD is a program to build and manipulate file lists. If you program in C, Pascal (Borland or Microsoft), MASM, COBOL, QuickBASIC, Fortran (Lahey or Microsoft), Intel 8096 assembler, Batch90 or DBC, then LISTBLD can create file lists automatically by scanning your source code for “include” statements (and COBOL “COPY” verbs). It can also use wild-cards and other file lists to add and remove files from file lists, and it can augment file list entries with version numbers or branch specifications for using TLIB with tree-structured library files.

dBase and Clipper programmers can generate TLIB-compatible file lists using dBFind, from The Software Development Factory, P.O. Box 1106-B, Hunt Valley, MD 21030. Tel: (410) 666-8129.

If LISTBLD does not recognize include statements in the programming language you use, please contact us. We will try to add support for your programming language.

Since it does not do a full parse of the source file, LISTBLD may occasionally make mistakes when scanning for include statements. For this reason, it echoes the include statements and file names to the console as it runs; if you see that it has deduced a file name incorrectly, you can manually edit the file list to correct the mistake, or use the LISTBLD REMOVE option to remove the erroneous file list entry, or simply ignore and tolerate the “file not found” error messages from TLIB. The usual cause for this is “commented out” include statements (or COBOL “COPY” verbs).

If your compiler allows “nested” include statements (that is, include statements within include files), then LISTBLD can be instructed to scan the nested include files, too, like this:

LISTBLD inputfile outputfile NEST

Otherwise, you would run it like this:

LISTBLD inputfile outputfile

Inputfile is normally the “main” (outer) program source file, and outputfile is often “*.lis”.

If there are several modules in your program, you can run LISTBLD once for each of them, using the ADD option to make LISTBLD add each set of new names to the same file list (outputfile), like this:

LISTBLD inputfile outputfile ADD
or
LISTBLD inputfile outputfile NEST ADD

Note that LISTBLD detects and deletes duplicate names, so outputfile will not end up with duplicate list entries even if two or more modules use the same include file.

To add file name(s) to a file list without scanning for include statements, you can use the ONLY option, like this:

LISTBLD inputfile outputfile ONLY
or
LISTBLD inputfile outputfile ONLY ADD

If inputfile is of the form @filelist, LISTBLD with the ONLY and ADD options can be used to merge two file lists together, eliminating duplicates.

Similarly, you can remove names from a list with the REMOVE option, like this:

LISTBLD inputfile outputfile REMOVE

By using wild-cards and file lists for inputfile, you can easily build file lists representing complex relationships between files. For instance, the following example builds a file list named Z.LIS containing all the assembler source files in the current directory plus all the source files for A.EXE and X.EXE, except for the standard include file STDIO.H and those include files used by B.EXE:

LISTBLD A.C Z.LIS NEST
LISTBLD X.C Z.LIS NEST ADD
LISTBLD *.ASM Z.LIS ONLY ADD
LISTBLD B.C TMP.LIS NEST
LISTBLD @TMP.LIS Z.LIS REMOVE
LISTBLD STDIO.H Z.LIS REMOVE
DEL TMP.LIS

The resulting file list can be used to specify input files for most programs in the TLIB package. For instance, to convert blanks to tabs in the files listed in Z.LIS, you could do:

MD TEMP
TABS IN @Z.LIS TEMP\*.*
COPY TEMP\*.*
ECHO Y | DEL TEMP\*.*
RMDIR TEMP

By default, LISTBLD records only file names. However, if you specify the RELATIVEPATHS option, it can also store “relative” paths along with the file names. The relative paths are relative-to the current subdirectory.

When the REMOVE option is specified without RELATIVEPATHS, then any relative paths are ignored, and LISTBLD removes every entry in the output file list which matches the specified input file(s).

When the REMOVE option is specified with RELATIVEPATHS, the complete path+name must match for each entry to be removed.

For example, suppose the EXCLUDE.LIS and MYFILES.LIS file lists are as follows:

exclude.lis	myfiles.lis
iostuff.c	aa\iostuff.c
f\analyze.h	iostuff.c
	bb\iostuff.c
	analyze.h
	aa\analyze.h
	compute.c
	f\analyze.h

Then specifying REMOVE without RELATIVEPATHS...

LISTBLD @exclude.lis myfiles.lis REMOVE

David Burton

2003-02-03T13:30:25

David Burton