by Lorne Walton, walton@telus.net
RATING: Intermediate
VERSION: FileMaker 5/5.5/6
PLATFORM: Macintosh, Windows
TECHNIQUE FILES: None
Did you ever wonder how much "metadata" (data about the data and non-data elements that make up a FileMaker database) is stored in a FMPro file? Not me. But now that I know a bit about it, I'm amazed by the way New Millennium Communications' new MetadataMagic pulls out that data.
NMCI adds to their fine stable of FileMaker add-ons with this new product, combining the power of plug-ins written in C and C++ with the ease of use of FileMaker Pro. Check it out at <http://www.nmci.com/>.
The free demo will process only 3 files at a time; the user can purchase licences to more capable engines, priced from $100 for a 5-file version all the way up to $700 for "unlimited files"; extra-cost options include cross-platform capability, redistribution licence and master password access.
Unlimited files? Apparently so: since the plug-ins open the files to be processed, rather than FileMaker itself, the 50-file limit has been bypassed. My advice: buy the most powerful licence you think you can afford; upgrade later, when the need arises. Just contact NMCI with credit card in hand, and request a new licence key.
The redistribution licence does not, of course, allow resale of the plug-ins that are the heart of MetadataMagic! What it does permit is distribution of MdM's output (the FMPro files) pertaining to the licensed developer's solution. This intensive developer-level documentation would assist the end-user who has purchased a master password and wishes to make modifications. Too, it would assure the customer that in the event of the developer dropping support for his package, the MdM files can aid a new developer in picking up that support.
Under the Hood
MdM includes 19 FMPro files, a trio of small html files, and a pair of plug-ins. The FMPro files act as the user interface and the vehicle for reporting the collected data. Most of the documentation is included as on-line help within the FMPro files. Click the "Additional Information" button from the main menu, before you do anything else.
Testing was performed using MetadataMagic version 1.0.2 on a 466MHz iBook under both OS 9.2.2 and OS X v10.1.5. [Version 1.0.3 has just been released, with a bug fix and an option for reporting in German.] My major experimentation involved a 24-file, 19MB solution that runs my office. The solution was in FMPro 5.5 format, and in good working order. Or so it appeared.
Getting started
All files to be processed must be organized into a single folder; subfolder contents will not be processed. This caused me some consternation, since my inclination is to break up my solutions into closely-clustered subfolders, and even refer "outside" to a couple of files that are shared with another loosely-related solution. It seemed more intuitive to want to test my solution under its "native" structure; dragging 24 files into one folder didn't feel like the right thing to do. But I did it, selected the target folder from MdM's main menu, typed a master password and a title for the diagnostic run, and it set to work.
MdM tracks its data-gathering into a log file as it works, in one of four modes, depending upon how detailed one wishes his feedback. Click "Process..." and then "Options" to choose among Terse, Concise, Verbose and Effusive logging, and also to set the FileMaker version and other preferences. Choice of mode does not affect the data gathered; only the log file differs.
On this same Options layout, one can choose to process files that are in FMPro 3/4 format. But note that FileMaker 5 or later is required to make MetadataMagic operate.
No user input is required during processing, and MdM can run in the background (albeit slower.) In "verbose" (a.k.a. "debugging") mode under OS 9, MdM took 22:24 to process all the files; the identical run under OS X took only 12:07. In the default "concise" mode, the same run under OS X required 10:25.
Considering the volume of data - i.e., metadata - which MdM sucks out of the subject files, those times seem highly respectable. The execution-time difference between OS X and OS 9 looks to be spurious, and may represent FileMaker having been inadvertently shuffled into the background during the run. On a subsequent run in OS 9 the elapsed times were 12:05 (verbose mode, 3209 records created in the log file) and 12:15 (concise mode, 512 records logged.)
Here's the beef!
Click on the link back to Menu, and then "Data Errors." 30 error records were created. Several of those merely indicated how many times a file had been closed improperly. Several more referred to "missing files" which I had not included in the test folder.
A pair of records noted that two files had each been "Recovered" once. Oops! Many developers believe that a FMPro file should not continue to be used once it has been Recovered by FileMaker. Instead, one should import the data into a recent "clean" backup, and rebuild any file structure modified since the backup.
With MetadataMagic, it's easy to find out whether a file has ever been recovered. Now if only MdM could tell us whether a file has been corrupted... but it can't get metadata out of FileMaker which isn't there.
Several error records pointed out some shoddy programming on my part, where a relationship was built between key fields that were not of the same data type. In almost every instance this was a "fixed link" relationship between two calculated fields, both having the value "1". Hmm... this seems to work okay... so far, but I must be more careful. I have amended my ways, using more descriptive field names (like "OneCalcText" instead of "One") in order to avoid this issue.
Each of those 30 error records includes the name of the file, script, or other element wherein the problem occurred. Click on the name of the object - colored and underlined like a hypertext link - and you're at a new screen showing details about the object, with the opportunity to tunnel deeper for the details.
Boldly go...
As you navigate among records, files and layouts, MdM's History file is keeping track of where you've been. A "bat-signal" logo in the upper left corner of most layouts allows navigation back and forward, as with an Internet browser. (The History feature does not work in multi-user mode. The user is invited to contact NMCI for a multi-user version.)
Go back to Menu, and choose the top right "Metadata..." button rather than "Data Errors." This time you see an array that displays all files that were processed, and beside each, the file's count of layouts, fields, relationships, file references, passwords, et cetera, and finally, the number of errors. Most of these items are underlined, again indicating a link where more detail may be found.
Click on the name of a file and see again the list of its element types (script, field, value list, group etc., everything you might imagine - except maybe graphics) Included are the facts that the file has 3,683 records and has been opened 1,764 times, for more than 32 million seconds. Click on "Calculations" and see a list of the calculation fields. Click on the name of a calc field, and view its definition, data type, and more.
How many of our FMPro solutions have unused/unreferenced elements? I have often scratched my head over whether it would be safe to trash an old script or a field. MdM gives you that piece of metadata as well. My solution contains 5 unreferenced layouts, out of 120 in total; somehow, I'm still wary of annihilating them. What does FileMaker's "unreferenced" metadatum really mean, anyway?
One has to wonder about some of the information FileMaker keeps. Like an array of pathname references dating back to when the solution was originally created. So this is how FileMaker goes looking for a referenced file. Wouldn't if be great if MdM could employ these pathname references in order to find all files associated with a solution, instead of requiring all files to be within a single, flat folder?
Each time MetadataMagic is run, it requires that the records from the previous run be expunged; this is scripted, of course. Somebody will want to keep back copies of his MdM runs; if so, compress the files before clearing them (encrypt them, since they contain your files' master passwords) and assign the archive an appropriate name, date, etc. Because the descriptive notes field is related to the other MdM files via filename and element, any notes you added will reappear in the proper context the next time MdM processes that file. Neat!
For the most part, it's only the most recent run that's important; after the solution files are modified, the previous set of metadata is obsolete. It's quick and effortless - if one keeps all referenced files within a single folder - to reprocess the files for an au courant report.
All layouts are attractively and intuitively designed. Several of the data-collection files have a Preview script, that displays the current set in a layout suitable for printing. Use MdM for a short while, and it's a breeze. But do read the built-in "Additional Information" for tips that make it all even more fluid.
Conclusion
MetadataMagic is pricey, but not out of line with competing products which may not do as much. It is a new, cool, powerful, easy to use, beautifully cross-linked, entry into the arena of FileMaker Pro diagnostic programs. It has the potential to facilitate and accelerate the development and troubleshooting of multi-file solutions. Download the files from NMCI and take a look. If you do serious database development, you owe it to yourself.
Happy FileMaking!
Lorne Walton is a fanatical FileMaker semipro developer who welcomes your beginner and intermediate-level questions. You may email the fanatic at walton@telus.net.