Louis Kessler’s Behold Blog

The Delphi programming language is having its #Delphi25 #Delphi25th birthday on Friday Feb 14, 2020. I’ve been using Delphi for about 23 years since 1997 when Delphi 2 was released.

Delphi is an amazing language. I use it now for Behold and Double Match Triangulator, and I’ve made use of it for a number of personal projects along the way as well.

It’s appropriate on this day that I write about Delphi and how I use it and what I like about it.

Pre-Delphi

I should provide a bit of background to give context to my adoption of Delphi as my programming language of choice.

As I entered high school (grade 10), my super-smart friend and neighbor Orest who lived two doors over and was two grades ahead of me recommended I follow his lead and get into programming at school. The high schools in Winnipeg at that time (1971) had access to a Control Data Corporation mainframe, and provided access to it from each school via a cardreader and a printer. You would feed your computer cards into the cardreader. In the room was one (maybe two) keypunches, likely KP-26 or maybe KP-29.

The computer language Orest used at the time and the school was teaching was FORTRAN, a Waterloo University version called FORTRAN IV with WATFOR and WATFIV. What an amazing thing. You type up a sequence of instructions on computer cards, feed them through the card reader, and a few minutes later your results are printed on classic fanfold computer output.

For three years of high school, my best friend Carl and I spent a lot of time in that small computer room together. I remember a few of the programs I wrote.

1. There was the hockey simulation following the rules of a hockey game we invented using cards from a Uno Card Game. We simulated a full World Hockey Association season of the 12 teams each playing 78 games giving each team a different strategy. 11 of my friends would each have a team and look for the daily results and standings.
2. For a special school event, my friend Carl and I wrote a dating program. We got everyone in school (about 300) and all the teachers (about 30) to fill out a multiple choice survey of about 10 questions about themselves, and the same questions for what they wanted in a date. During our school event, people would come to the computer room, and Carl and I would run them against the database and give them their top 5 dates with hilarious results.
3. I played touch football with a number of friends once or twice a week during the summer. I recorded all the stats on the back of a computer card in between plays, and I then would punch the results onto computer cards and wrote a program that would give total all the passing stats, receiving stats, interceptions and fumble recoveries by player, giving the leaders and record holders in each category. Everyone loved seeing the stats and played harder and better because of it.
4. I wrote a program to play chess. Carl wrote one as well. We had a championship match – chess program vs chess program that got us in our city’s newspaper.

At University, I took statistics but also included many computer science courses. While there, I continued work on my chess program in my spare time and the University of Manitoba sponsored me as a contestant in the North American Computer Chess Championships in Seattle, Washington in 1977 and in Washington, D.C. in 1978. Games were played with modems, and connected dumb terminals to the mainframes back at our Universities. Read all about my computer chess exploits here: http://www.lkessler.com/brutefor.shtml

After getting my degree in statistics, I went for my Masters in Computer Science. Now we finally no longer needed computer cards, but had terminals we could enter our data on. There was a Script language for developing text documents, and I used it to build my family tree, with hierarchical numbering, table of contents and an index of names. It printed out to several hundred pages on fanfold paper. I still have that somewhere.

I started working full time at Manitoba Hydro as a programmer/analysis rewriting  and making enhancements to programs for Tower Analysis (building electric transmission towers) and Tower Spotting (optimizing the placing of the towers). These were huge FORTRAN programs containing tens of thousands of lines of what we called spaghetti code.

Then I was part of a 3 year project to develop MOSES (Model for Screening Expansion Scenarios) which we wrote in the language PL/I. That was followed by another 3 year project from 1986 to 1988 where our team wrote HERMES (Hydro Electric Reservoir Management Evaluation System) which we also said stood for Having Empty Reservoirs Makes Engineers Sad. I learned that one of the most important parts of any program is coming up with a good name for it. I also learned how to three-ball juggle.

The HERMES program was written in Pascal. That was a new language for me but I learned it quite thoroughly over the course of the project. I believe I purchased my first personal computer, a 286 PC 12 Mhz for home sometime around 1990. When I did, FORTRAN was still available but very expensive. So instead I purchased Borland’s Turbo Pascal.  I started programming what would one day become my genealogy program Behold.

My Start with Delphi and Evolution Thereof

I like to joke that I’m not an early adopter and that’s why I didn’t buy into Delphi when it came out in 1995, but did buy Delphi 2 in 1997. Delphi was basically still the Pascal language. But what Delphi added over Turbo Pascal was primarily two things:  the addition of Object- Oriented Programming (OOP), and an Integrated Development Environment (IDE). Those were enough that I had to go “back to school” so to speak, and I loaded up on getting my hands on any Delphi Books that I could. They’re still on my shelf now.

I purchased Delphi 2 on May 14, 1997 for $188.04 plus $15 shipping & handling.

I didn’t upgrade every year. It was expensive. But I only upgraded when I felt there was some important improvement or new features I needed.

I upgraded to Delphi 4 in June 1998 for $249.95 plus $15 s/h. At this time, Borland had changed its name to Inprise. By 2001, they abandoned that name and went back to Borland.

I was able to use Delphi 4 for quite some time. Finally there was a feature I absolutely needed and that was Unicode which came in Delphi 2009.  I was allowed to upgrade my version of Delphi 4 and I did that and upgraded to Delphi 2009 in Sept 2008 for $374.

Embarcadero purchased Delphi from Borland in 2008. In 2011, I upgraded to Delphi XE2 for $399 which included a free upgrade to Delphi XE3.

I upgraded to Delphi XE7 in 2015 for $592. And I upgraded to Delphi 10.1 in 2016 for $824.40.

The upgrades were starting to get expensive so in 2017 I started subscribing to Delphi maintenance for $337 per year.

Third Party Packages

Delphi includes a lot of what you want, but not everything. I needed a few packages from third parties who built components for Delphi. For Behold I used two:

TRichView by Sergei Tkachenko. TRichView is a WYSIWYG (What You See Is What You Get) full featured document editor that forms the main viewing (soon to be editing/viewing) window of my program Behold that I call “The Everything Report”. Behold is listed among the many Applications that have been made with TRichView.

I purchased TRichView in 2000 when it was still version 1.3.  Now it’s 7.2. Back then the cost was $35, and it was a lifetime license that Sergey grandfathered in for his early customers. He has continued to develop the program and has not charged me another nickel for any upgrades. I did, however, pay $264 to Sergey in 2004 for some custom code he developed that I needed. I liked that lifetime license policy so much that it inspired me to do so as well for my Behold and Double Match Triangulator customers who all get free upgrades for life when they purchase a license. Sergey no longer offers lifetime licenses. His current price for TRichview is $330, but he also offers other products that work with it. That’s at 20 years of Delphi development for Sergey.

LMD Innovative’s ElPack is the other package I use for Behold. This is a package of over 200 components that extend the functionality of the VCL controls that Delphi has. The main purpose I purchased this was for their ElXTree which allows custom creation of TreeViews and grids:

I first purchased ElPack in 2000 from the company EldoS (Eugene Mayevski) who originally developed it.  The cost was $68. About 6 months after I purchased it, I noticed a free product available called Virtual Treeview written by Mike Lischke, but I was already using and happy with ElPack so I continued to use it. I considered switching to Virtual Treeview several years later, but my use of ElPack was already so deeply embedded into Behold, that it wasn’t worth the effort.

I did have to pay for upgrades to ElPack, so I upgraded only when there was a reason to. Usually it was because I got a new version of Delphi and the old version wouldn’t install. Also, my third party packages were also a reason I didn’t upgrade Delphi so often, because I couldn’t really upgrade until both TRichView and ElPack had versions that worked with the new version of Delphi, which could take up to a year after the Delphi version release.

In 2003, LMD Innovative acquired ElPack from EldoS and continued developing it. LMD’s current price for ElPack is $190. They have a partnership with TRichView and give me 20% off for being a TRichView customer. I tend to upgrade ElPack every two years or so.

TMS Software’s FlexCel Studio was a package I purchased for Double Match Triangulator (DMT) to provide native Excel report and file generation, without requiring use of Excel automation and not even requiring Excel on your computer. I use it to produce the Excel files that DMT puts its results into. The capabilities of this component actually amaze me. It can do anything you can think of doing in Excel and more.

I first purchased FlexCel in August 2017 for $157.

Additional Tools I Used to Work With Delphi

Developing programs with Delphi requires additional tools from time to time. Here’s some of the tools that were useful in my Delphi Development:

In 2009, I purchased for $129 a program called EurekaLog, which integrated with Delphi and worked to find and help locate memory leaks in my program Behold. The program helped me determine how my code was causing leaks, so after a few years and all leaks eradicated and better programming to avoid future leaks, I really didn’t have a great need to keep using the program.

In 2010 when I was tuning Behold for efficiency, I purchased a line by line profiler from Automated QA called AQTime that worked by itself or with Delphi. This was a very expensive program at $599, but I was able able to speed up Behold 100 times by finding inefficient algorithms and sections of code that I could improve, so it was worth the price. The program has since been acquired by SmartBear and still sells for $599. I no longer have a version of this program that works with the latest version of Delphi. Delphi does provide a lite version of AQTime for free, but that does not include its fantastic line-by-line profiler. I’m no longer in need of super-optimizing my low-level code because that rarely changes. When I need to ensure a section of code is not too slow, I now put timers around the section and that often tells me what I need to know.

Dr. Explain is the program I chose for writing the help files for my programs. I first purchased it in 2007 for $182, upgraded in 2014 for $100. The current price of an advanced license is $390.

And my installation program of choice for Behold and DMT is the free Inno Setup Compiler from jrsoftware. I purchase Comodo Code Signing certificates for about $70 a year.

Personal Uses of Delphi

Other than the two programs Behold and DMT that I am developing and selling licenses for, I also have used Delphi over the years to build some programs for my own use. These include:

• A database search program I build for my local Heritage Centre so they could easily query their Microsoft Access databsse which had listings and details of over 60,000 items. Originally written in Turbo Pascal and later converted to Delphi. (1996)
• A program to build some of my link web pages for me such as my Computer Chess Links page. (1997)
• A program to screen stocks for me to find stocks that I was interested in purchasing. (1997)
• A program to run through all possible picks and determine what selections my competitors picked in our local newspaper’s hockey, football and stock market contests (1997). (Aside:  I have won more than $20,000 in such contests using this type of analysis to help me gain an advantage.)
• A page counter for early versions of my websites. (2001)
• A program to help win at the puzzle called Destructo, where you’re trying to break through a wall. (2001)
• A program that produces the RSS feeds for this blog on this website (2004).
• A program to analyze the log files from my websites, especially to find pages that link into my sites.(2005)
• A program to help play sudoku.  (2005)
• A program to download stock market data and do technical analysis for me. (2008)
• A program to analyze 100 GB raw data files from whole genome DNA tests. (2019)

One thing I never have done is resurrected my chess program. For a while I considered it, but I knew it would be a lot of work and I didn’t want to take my time away from my genealogy software. In the past couple of years, deep learning and Alpha Zero has made all other programs irrelevant.

What’s Next with Delphi

I am very pleased that Embarcadero has continued to support and improve Delphi and that my Third party packages continue to roll. Hopefully that will continue for the foreseeable future.

The stability I’ve had over the past 24 years being able to use Delphi has been fantastic. The development environment is great. I love how fast it compiles, how fast the code runs, and how easy it is to debug.

Here’s 24 Years of Delphi and 25 Years of Excellence and here’s Going Forward.

On my speaker topics page, I like saying that “Louis is fluent in five languages: English, Delphi/Pascal, HTML, GEDCOM and DNA.”

Well now I better post this page and get to bed, because I have to be up in 9 hours for the Happy Birthday Delphi 25 celebrations

I’ve working hard to get Behold 1.3 completed. It will primarily be a newer iteration of Behold’s Everything Report. Once that is released, I’ll start my effort to add GEDCOM export followed by editing.

I’ve designed Behold to be a comprehensive and flexible GEDCOM reader that understands and presents to you all the data contained in GEDCOM of any flavour, from 1.0 to 5.5.1 with developer extensions and user-defined tags. So when John Cardinal came up with his GEDCOM Assessment site, that was a opportunity I couldn’t resist.

“assess.ged is a special GEDCOM file which you may use to perform a review of the GEDCOM 5.5.1 import capability of any program that reads a GEDCOM file and imports the contents”

John is a long-time user of The Master Genealogist program written by Bob Velke. John is also a programmer and wrote programs to work with TMG including Second Site for TMG, TMG Utility and On This Day.

After TMG was retired in 2014, John wanted to help people get all their data out of TMG allowing them to transfer to other programs so he wrote the TMG to GEDCOM program. He also wrote a program that creates an e-book from a GEDCOM file called Gedcom Publisher. And John then wrote a program to create a website from any generic GEDCOM file and called that program GedSite.

In the process of all this, John gained an expertise in working with GEDCOM and has made tests for GEDCOM compatibility that he invites all genealogy software authors to try.

So try it I shall. 

I followed John’s “process” and downloaded version 1.03 of assess.ged file as well as the images file references and placed the latter in a C:GedcomAssessment folder. Then I loaded assess.ged into Behold 1.2.4 and used his website’s Data Entry page to capture the results. This really is a beautifully set up assessment system. My complements to John Cardinal.

A Few Things To Fix for Version 1.3

There were a number of tests that illustrated some aspects of GEDCOM that Behold does not fully support. I’ve made a list of them here:

1. Behold by default uses the first person in the file and treats that person (and their spouse(s)) as the family the report is about. (You can of course pick anyone you want instead of or in addition to the first person). The assess.ged file does not link the 185 people in the file to each other, except for two who are connected as spouses. Behold was not using the first person in the file as a singular family but instead had the first section blank and listed all the people, including that first person, in its “Everyone Else” section. This should be a simple fix.
2. I was surprised to see Behold display:  1 FACT Woodworking 2 TYPE Skills as Woodworking Skills rather than Skills: Woodworking. That’s a bug because I intended it the latter way. Same for 1 IDNO 43-456-1899 2 TYPE Canadian Health Registration which was being displayed as 43-456-1899 Canadian Health Registration rather than Canadian Health Registration: 43-456-1899.
3. Behold somehow was ignoring and not displaying the TIME value on the change date of a record.
4. The CONC tag to concatenate two lines is specified by GEDCOM to require the last word in the first line be split so that it’s second half begins the second line. Behold does this, but in doing so, Behold trims the lines before concatenating. As a result, if a GEDCOM used a non-standard method of including a leading space on the second line or a trailing space on the first line, then it is ignored and the word at the end of the first line and the beginning of the second line would be joined with no intervening space. I haven’t noticed programs using this non-standard format, but even so, I’ll think about it and maybe I’ll remove Behold’s trimming of concatenated lines in version 1.3.
5. Behold displays: “Birth, date”.  But it should display “Birth: date”. Same for other events such as “Adoption, date” or “Baptism, date”. How did that ever happen?
6. Behold currently displays the user-defined tag _PRIM as “Primary: Y” after a date, but retains the first-entered date as primary and does not use this tag to make that date primary. I think about deciding to honor the _PRIM tag in version 1.3.
7. The non-standard shared event tag, e.g. 1 CENS 2 _SHAR @I124@ is not being displayed correctly by Behold. This will be fixed.
8. Behold does not convert @@ in notes or text values to @, as it should. Technically all line items should be checked for @@ and changed as well so that includes names.
9. Hyperlinks to objects unfortunately do not open the file because Behold added a period to the end of it. This is a bug that I noticed a few weeks ago and has already been fixed for the upcoming version of Behold under development.
10. Alias tags (ALIA) whose value is the name of the person rather than a link is valid according to the GEDCOM standard, but it may be something I want to support if I see it was exported into GEDCOMs by some programs.
11. I’m not displaying the tags under a non-standard top level 0 _PLAC structure correctly. This includes 1 MAP, 2 LATI, 2 LONG and 1 NOTE tags under the 0 _PLAC record.
12. Non-standard place links such as: _PLAC @P142@ that link to the 0 _PLAC records should have been working in Behold, but the display of these links needs to be improved.
13. If a person’s primary name has a name type, then it should be repeated with the type on the next line, e.g.
       Birth name:  Forename(s) Surname
    Also additional names should be called “Additional name” rather than just “Name”.
14. Names with a comma suffix should not be displayed with a space between the surname and the comma. I’ve actually never seen this in the wild.
    e.g. /Smith/, Jr should be displayed as Smith, Jr and not Smith , Jr
15. Notes on places are repeated and shouldn’t be.  Dates should be shown following any notes or other subordinate info for the place.
16. Addresses could be formatted better.
17. EVEN and ROLE tags on a source citation should have their tag text looked up and displayed instead of just displaying the tag name.
18. The OBJE link was not included in source citation when it should have been.

So that was a really good exercise. Most of these are minor, but a lot more issues came up than I expected. Over the next few days, I’ll resolve each of these in the development version of Behold which soon is to become version 1.3.

  
Results and Comparison

John presents a Comparison Chart that currently compares the results for 17 programs. There are 192 tests. Here’s my summary of John’s Comparison.

I’ve added Behold’s result in my chart. I’ve also excluded John’s program GedSite in summarizing the other programs, because his results are for a program that has already been tuned to handle these tests. So GedSite’s numbers are a good example of the results that I and other developers should try to attain with our programs.

Behold didn’t do too bad with 161 supported constructs out of 192. Best was GedSite’s 185 followed by Genealogie Online’s 179, then by My Family Tree’s 169 and then by Behold’s 161. Genealogy Online is the baby of Bob Coret who is another GEDCOM expert, and My Family Tree is by Andrew Hoyle of Chronoplex Software who also makes GEDCOM Validator, so you would expect both of them to be doing well with regards to GEDCOM compliance.

I’ve emailed the JSON text file of Behold’s results to John. Hopefully he’ll add Behold to his comparison chart.

Comments About John’s Test File and Data Entry Page

1. The assess.ged file version 1.03 includes a 1 SEX M line in each of the test cases. I’m not sure why. SEX is not a required tag in an INDI record. For a test file, it would be simpler to just leave the SEX lines out.
2. I disagree with the constructs of two of the Master Place Link by XREF tests. They include within one event, both a standard test place link and the non-standard place xref link, i.e.: 
      1 CENS
      2 PLAC New York
      2 _PLAC @P158@
   The trouble I have with this is that GEDCOM only allows one place reference per event. By using this alternative tag, you’ve effectively got two which is illegal if they were both PLAC tags. And what if they are not the same? John should take out the 2 PLAC New York line from his NAME 02-Link by XREF tests where he has the 2 _PLAC tag so that there is only one place reference. Any programs allowing both PLAC and _PLAC tags on the same event should cease and desist from doing this. The second test where the 3 _PLAC tag is under the 2 PLAC tag is an even more horrible construct that no one should support.
3. The GEDCOM Assessment Data Entry Page does not completely function in all browsers. When using my preferred browser Microsoft Edge, entering “Supported (w/comment)” did not bring up the box to enter the comment. I tried Internet Explorer and the page did not function at all. I had to switch to Google Chrome (or Firefox) to complete the data entry.

Conclusion

What this little exercise does show is how hard it is to get all the little nuances of GEDCOM programmed correctly and as intended. This assessment took the better part of a day to do, but I think it was well worth the time and effort.

And what’s really nice about having a file with test cases is that they provide simple examples that illustrate issues that can be fixed or improved.

I hope all other genealogy software authors follow my lead and test their programs with GEDCOM Assessment’s assess.ged file. Then it’s a matter of using this analysis to help make their programs more compatible with the standard and thus do their part to help improve genealogical data transfer for everyone.

Update  Feb 10:  John reviewed the assessment with me. A few results changed status and I’ve updated the table above. John mentioned that his creation of GedSite wasn’t a conversion of Second Site for TMG, but was a completely new program.

Behold version 1.2.4’s final assessment is now available here:
https://www.gedcomassessment.com/en/assessment-behold.htm

Once I complete version 1.3, I’ll likely submit it again for a new assessment.

Update: Sept 2024: Sadly, John’s site is gone. But it still exists on Archive.org and I’ve updated the links above.

The Family History Fanatics’ @FHFanatics online Winter DNA eConference has just finished and it was so much fun. Andy and Devon sure know how to put un a good show.

This was the first time I’ve presented at an online conference.  I was able to do this comfortably from my office at home and my family was really good and went out of their way not to bother me for 6 hours.

Devon Noel Lee, Jonny Perl and Paul Woodbury also presented and it was so great getting to interact with them. It was the next best thing to being at a physical conference, without the need to spend time in airports and hotels.

In addition to all the great genetic genealogy methodologies presented, I also learned that I know nothing about what genes are on what chromosomes, and that audiences love to suggest random numbers.

We ended with a question/answer period that turned into an entertaining roundtable discussion.

Now to get back to work and figure out how Jonny and I are connected through the 16 cM segment on chromosome 7 that we share.