Louis Kessler’s Behold Blog

FamilySearch released the official version of GEDCOM 7.0 today. You can find it at https://gedcom.io selecting “Specifications” from the menu.

The standard is available as PDF or as HTML pages. The PDF is 96 pages, compared to Version 5.5.1 which was 101 pages. The specifications page also has a link to the Github Repository which is where the people on the team have been putting together the document. There’s also a Changelog, as well as the specifications for the previous version 5.5 and 5.5.1 of GEDCOM and for GEDCOM X which programmers primarily use to transfer data directly with FamilySearch’s Family Tree.

The FamilySearch GEDCOM Specification itself lists its version as 7.0.1.  This is likely due to a GEDCOM 7.0 being temporarily released as Release Candidate 7.0.0-rc1 back in February for a week during RootsTech before it was pulled, likely deemed not-ready.  So the 7.0.1 differentiates this release unambiguously. See my blog post GEDCOM 7.0 from February which explains what happened with that release candidate.

This version is called 7.0 and not 6.0 because there was a GEDCOM XML 6.0 draft issued on Oct 2, 2000, an updated draft on Dec 28, 2001, and a GEDCOM 6.0 XML Beta Version issued on Dec 6, 2002. The GEDCOM XML 6.0 specifications never became official and I don’t believe any program implemented it. It evolved into GEDCOM X which FamilySearch released instead.

Contributors

The Contributors are listed on page 91 of the specifications. They include a wide range of technical people and genealogy software developers. The contributors listed are:

Managing Editors

• Gordon Clarke, FamilySearch
• Luther Tychonievich, FHISO and University of Virginia

Taskforce

• David Pugmire, FamilySearch
• Jimmy Zimmerman, FamilySearch
• Larry Telford, FamilySearch
• Matt Misbach, FamilySearch
• Russell Lynch, FamilySearch
• Robert Raymond, FamilySearch
• Gaylon Findlay, Ancestral Quest
• Derek Maude, Ancestry
• James Tanner, The Family History Guide
• John Cardinal, Family History Hosting
• Albert Emmerich, GEDCOM-L
• Dave Berdan, Legacy Family Tree
• Evgen Zherebniy, Software Mackiev
• Jason Fletcher, Midlera Software
• Uri Gonen, MyHeritage
• Dallan Quass, OurRoots.org
• Tony Proctor, SVG Family-Tree Generator
• Bill Harten, Puzzilla
• Bruce Buzbee and Mike Booth, RootsMagic

Development Teams

• Tags team: Luther Tychonievich, Albert Emmerich, Russell Lynch, Tony Proctor, John Cardinal
• Extensions team: Luther Tychonievich, Tony Proctor, Jimmy Zimmerman
• Notes team: Dallan Quass, David Pugmire, Jason Fletcher, Russell Lynch
• External Media team: Dallan Quass, Jason Fletcher, Derek Maude

I know there are also many others not listed who contributed as well (myself included) as FamilySearch has been reaching out to all developers for their input.

Mondays with Myrt

This morning, Dear Myrtle (aka Pat Richley-Erickson), had her Mondays with Myrt session. I usually come to those. In her session, Tony Proctor was in attendance and while the MWM session was going on, Tony got an email announcement that GEDCOM 7.0 was released and mentioned it, so Pat invited both Tony and me onto the panel to talk about the specs from a developer’s point of view. This morning’s session is available on the Dear Myrtle blog:  ARCHIVED: Mondays with Myrt – 7 June 2021 including GEDCOM 7.0 info.

I should mention here (and I mentioned it live in the session) that Pat and her cousin Russ were the ones who originally started the BetterGEDCOM project about 10 years ago.
   

The BetterGEDCOM project got developers to discuss their likes, dislikes and recommended changes to the GEDCOM standard that was at that time over 10 years old. Too much data, especially sources, were not being transferred correctly or at all between different genealogy programs. A lot of excellent discussion took place, but Pat realized there needed to be an official organization to do this work.

So Pat spearheaded the creation of FHISO.

FHISO was created to be a standards-setting organisation for genealogical information. FHISO stands for Family History Information Standards Organisation. Their website is at: https://fhiso.org/

FHISO got buy-in from many genealogy software developers and solicited input from all developers. They had a Call for Papers that received dozens of submissions including two from me.

A number of people were on the FHISO Board and it had several chairs, the most recent being Luther Tychonievich, a Computer Science professor at the University of Virginia.  Luther over the past few years worked with FamilySearch as a managing editor to help get GEDCOM 7.0 released.

So it really is Pat Richley-Erickson and Russ Worthington who deserves a big thanks. It’s their initiative that ultimately led to this.

So Now What Happens?

Well, in my opinion, I think it will take a while for Version 7.0 of GEDCOM to be implemented by genealogy software developers. There are a lot of changes and it will not be simple to implement.

During the Dear Myrt session, Randy Seaver told us that RootMagic said they’ll have GEDCOM 7.0 implemented in a month. Personally I highly doubt it. They are currently still working to get RootsMagic 8 released and they are behind schedule. I’m sure they won’t want to mess with GEDCOM and delay their new version any more.

Changing GEDCOM input to support 7.0 would be relatively easy, if it weren’t for the new constructs needing support that a program may not yet have in their database. Export is trickier and needs to be carefully and precisely done, and avoiding custom tags where constructs are available so that other programs will be able to read their data.

FamilySearch is making a few tools available to help developers. Currently they have a v5.5.1 to v7.0 Conversion Code tool. Developers will be able to use this to convert their own GEDCOM exports and see what they will need to import and what their export will have to look like.

The Conversion Code Repository also includes parser/js, a simple program to read GEDCOM 7.0. It is written in JavaScript, so developers using other languages will have to translate it to their own language. But most developers already have their own GEDCOM 5.5.1 parser, so they’ll likely better off just carefully updating their own code.

I am impressed by the small size of the javascript parser. It is only 67 lines, and that include comments! Hard to believe it truly can do all the conversion. Has anyone tested it?

It won’t really help you if your genealogy program’s the first to support GEDCOM 7.0. You’ll need to wait until a number of different programs support it before you can use it to transfer your data from one program to another. More importantly, people today usually have a desktop program and an online systems they use. You’ll need to wait until both your desktop program and the online system (Ancestry, MyHeritage, WikiTree, Geni, etc.) support GEDCOM 7.0 before it become useful to you.

That might start happening in the next year or so. If more developers adopt GEDCOM 7.0, it will gain more traction as a standard, and that will encourage other developers to use it to replace 5.5.1.

My program Behold is currently just a GEDCOM reader. If GEDCOM 7.0 comes to be used by a number of different programs, then I’ll add GEDCOM 7.0 input into Behold. I’ll be looking at the various implementations of GEDCOM 7.0 export from the programs that first implement it and make sure I can handle them, as I’m sure there will still be significant differences in how different developers interpret and implement it.

I’ve been an Independent Software Vendor (ISV), i.e. single developer operation, for over 15 years. I have had my products Behold and Double Match Triangulator available for download for the many versions of each of them.

I never put adware, spyware, viruses or anything bad in my programs. I pay money each year to code sign them so that users and Windows itself can be assured that the program they are installing is the one that I distribute and has not been modified by anyone else.

From time to time, one specific antivirus tool has a false positive with one of my programs and declares it bad. When a user tells me about that, it’s usually not difficult for me to go to the antivirus company’s website and fill out a form to ask them to check my program. They’ve never taken very long to whitelist my program and the problem is solved.

Microsoft Smart Screen

Not too long ago, Microsoft made enhancements to it’s SmartScreen component that it uses on Windows 10 to help protect users against potentially malicious software. Up until a few months ago, the code-signing I added to my Behold and Double Match Triangulator setup programs was good enough for SmartScreen to accept them.

But over the past few months, the following has started to happen. Clicking on the downloaded setup program for either Behold or DMT now pops up the following window which is reminiscent of the dreaded blue screen of death:

Notice there is only one option: “Don’t run”.

You have to know enough and be willing to click on the non-obvious “More info” link to have it allow you to execute the program:

This will display the app information, and they know that I am the publisher because the program is code signed.

And now a “Run anyway” button is available.  Yeah, right. "Run anyway” sounds reassuring – not!

Clicking on “Run anyway” now takes you to the User Account Control window, which is the standard Windows safety procedure for any installation program, and at this point everything is normal:

(I had to take a photo of this window, because it takes over your screen until you answer Yes or No and you can’t screen capture it.)

Smart Screen and Microsoft Edge

Even worse, if you are using Microsoft Edge, then it gives you several additional ominous warnings. First, when you click the download link:

“behold-setup.exe was blocked because it could harm your device” – is not a very friendly download message.

If you hover your mouse over the download box, you get this:

It now shows a garbage can, so that you can delete the download, and three dots. This obviously is more suggestive that you click on the garbage can rather than the three dots.

If you do decide to click on the three dots, you get this:

Once again the first option is Delete. The second is Keep.

“Report this file as safe” takes you to a page where you can report the download as safe, either as a user, or as the developer (see below in this post).

“Learn more” takes you to a page about Microsoft Defender Smart Screen.

“See more” just lists your other downloads.

Clicking on “Keep” brings up yet another scary warning:

So how do you feel about installing a program which “might harm your device” when the only two options appear to be Delete and Cancel.

You have to know enough to click on “Show more” to proceed with the download. Doing so will give you this:

Notice they display my personal information. That is available in my setup program because it is code-signed. They pull out that information and don’t even display it well, showing my name twice, and giving my personal address (not nice of them) when my name, email address, company name (I use Behold Genealogy) and website address would have been much better.

You have to then realize that you must click on “Keep anyway”, and the setup file will finally appear in your Downloads with the ability to Open (i.e. run) it:

If you then try running it, you will get the blue “Windows protected your PC” box described earlier.

So that’s 3 additional ugly warnings that Microsoft Edge adds as obstacles to ensure that you really want to install my program. That even scares me and I know that my program is safe, so I imagine it must stop everyone else in their tracks preventing who knows how many people from wanting to try my programs.

Having my code signed used to be enough for Edge and it always had earlier allowed the download without problems. Try downloading in Google Chrome or Firefox and there are no obstacles. Installation still gives the blue “Windows protected your PC” box, but that’s a Windows thing, not a browser thing.

Report this file/app as safe

Clicking on one of the “Report … as safe” links takes you to this page:

There are two options, to report a program safe as a user, or as the developer.

Clicking the first button results in this:

Clicking on the 2nd option as the owner expands that form to ask more information:

I have submitted this several times in the past few months for both my programs.

After submitting this form as a developer, I get this email back:

And the next day, I get a similar email saying “Your file has been analyzed”. I click on the “View your submission” link in it and it takes me to this page:

If I read that correctly, they seem to be confirming that :

“behold-setup.exe has since established reputation and attempting to download or run the application should no longer show any warnings”.

They then say the signing certificate is still establishing reputation. That should be for other programs I sign it with. Behold and DMT should now be okay.

But they aren’t. Downloads still are going through all the above rigmarole.

I’d be okay if this procedure worked and cleared my program. Unfortunately it did not. I can keep trying, but this is more than frustrating I have to say.

Extended Validation (EV) Code Signing.

There does appear to be one way to get rid of these horrendous messages. That is to upgrade my level of Standard Code Signing to Extended Validation Code Signing.  For only $200 more a year, I can get an EV Code Signing certificate:

Supposedly from what I read, an EV Code Signing certificate will alleviate all these SmartScreen warnings as soon as I start using it.

So a week ago, I purchased a new EV Code Signing for 3 years. For some reason, they were taking a long time to validate my phone number. Yesterday, I found that EV certificates are only available for businesses, and they confirm your signer information by verifying that your organization is valid and that the phone number corresponds with the organization.

I call my software development “Behold Genealogy” but it is not an incorporated business. I declare my earnings on my personal income tax every year. For me to register Behold Genealogy as an official business involves many complications and is more than I want to do.

I’ve now sent in my EV Code Signing refund request.

So What to Do?

As an Independent Software Vendor (ISV), I am somewhat screwed here. Microsoft is putting up roadblocks to my users making them distrust my programs so that they’ll be reluctant to download and install them. And the reputation of my programs is only able to get better if there are users downloading my programs, A bit of chicken and egg here.

All I can really do is add some explanatory info at my two download links to help assure downloaders, try to get their trust, and give them instructions on how to avoid the obstacles and download my programs.

And I hope that Microsoft takes a closer look at the problems this is causing to ISVs like me, and at least mean it when they say my programs have now “established reputation”.

I am very relieved that this happened on its own in less than a month.  My Code Signing Certificates should now retain their reputation at least until I have to renew them which is 2 and a half years from now.

It would have been nice to have received notice when my “reputation” was achieved. I had been working on other ways to package my software so that Microsoft might approve it. e.g. build it as an MSIX package and submit it to the Microsoft Store, but even that wouldn’t have guaranteed anything.

None-the-less, this still was not a nice thing for Microsoft to do, and for the past month (and I’m not sure how many before that), it caused me a lot of anxiety and unnecessary effort to attempt to find a solution. If Microsoft’s “Report a Download” page resulted in a confirmation of established reputation and no more warnings that actually was true at the time, and not a month later, I would feel much better about it.

Randy Seaver’s Saturday Night Genealogy Fun challenge for this week is:

I thought I’d do that, following Randy’s template:

A) On AncestryDNA, I have 149,270 matches. My Top 10 matches are:

* First cousin, mother’s side, 1,047 cM, in tree
* First cousin, mother’s side, 914 cM, in tree
* First cousin 1x removed, mother’s side, 411 cM, in tree
* Second cousin 1x removed, father’s side, 355 cM, in tree
* Second cousin, father’s side, 202 cM, in tree
* Second cousin, 1x removed, father’s side, 183 cM, in tree
* Second cousin, mother’s side, 162 cM, in tree
* Second cousin, mother’s side, 159 cM, in tree
* Unknown cousin, father’s side, 122 cM, not in tree
* Unknown cousin, father’s side, 117 cM, not in tree

I have 10 other relatives in my tree (total 18) who tested at Ancestry.
I have 149,252 matches who are unknown cousins who are not in my tree.

B) On 23andMe, I have 1,501 matches. My Top 10 matches are:

* First cousin 1x removed, father’s side, 7.36% (548 cM), in tree
* First cousin 1x removed, father’s side, 3.67% (273 cM), in tree
* Second cousin, mother’s side, 3.13% (234 cM), in tree
* Second cousin, father’s side, 2.13% (159 cM), in tree
* Second cousin, father’s side, 2.00% (149 cM), in tree
* Second cousin, father’s side, 1.83% (136 cM), in tree
* Second cousin, father’s side, 1.69% (126 cM), in tree
* Unknown cousin, father’s side, 1.64% (122 cM), not in tree
* Unknown cousin, father’s side, 1.50% (112 cM), not in tree
* Unknown cousin, unknown side, 1.47% (110 cM), not in tree

I have 8 other relatives in my tree (total 15) who tested at 23andMe.
I have 1,486 matches who are unknown cousins who are not in my tree.

C) On MyHeritageDNA, I have 18,020 matches. My top 2 matches are:

* Uncle, father’s side, 1,994 cM, in tree
* First cousin 1x removed, mother’s side, 466 cM, in tree

The next 8 matches are all unknown cousins not in my tree with cM:
141, 127, 124, 123, 119, 116, 112, 112.

I have 1 other relative in my tree (total 3) who tested at MyHeritage.
I have 18,017 matches who are unknown cousins who are not in my tree.

D) On FamilyTreeDNA, I have 25,672 matches. My top 2 matches are:

* Uncle, father’s side, 1,861 cM, in tree
* First cousin 1x removed, mother’s side, 442 cM, in tree

The next 8 matches are all unknown cousins not in my tree with cM:
192, 172, 168, 164, 163, 163, 162, 161.

I have 1 other relative in my tree (total 3) who tested at FTDNA.
I have 25,669 matches who are unknown cousins who are not in my tree.

E) On LivingDNA, I have 2,228 matches. My uncle shares 2,003 cM.
The next 9 matches share:  145, 131, 130, 129, 123, 121, 120, 118, 116.
I have 2,227 matches who are unknown cousins who are not in my tree.

F) On GEDmatch, using Tier 1 One-to-Many, I have 59,415 matches that are 7 cM or larger. My top 3 matches are:

* Uncle, father’s side, 1,918 cM, in tree
* First cousin, mother’s side, 958 cM, in tree
* First cousin 1x removed, mother’s side, 433 cM

The next 7 matches are all unknown cousins who are not in my tree with cM: 104, 94, 92, 87, 87, 84, 84.
I know I have 1 other relative in my tree (total 4) who uploaded to GEDmatch,
but I have not thoroughly checked the other 59,405 matches to see if there are any others.

G) On Geneanet, I have 763 matches. My uncle shares 1,925 cM.
The next 9 matches share: 55, 54, 53, 51, 50, 49, 49, 47, 46.
I have 762 matches who are unknown cousins who are not in my tree.

H) On Geni, I have 1,857 matches. My top 10 matches share:
164, 163, 152, 148, 147, 144, 138, 138, 135, 135.
All 1,867 matches are unknown cousins who are not in my tree.

I) At Borland Genetics, I have 862 matches. My top 3 matches are:

* my father, 1887 cM, in tree
* my mother, 1887 cM, in tree
* my uncle, 1805 cM, in tree

My next 7 matches are unknown cousins who are not in my tree with cM:
74, 61, 54, 48, 48, 47. 40. 38

Note that my father and mother’s DNA were partly assembled from my and my uncle’s DNA using the Borland Genetics tools.

What does this indicate?

* With endogamy, you get a lot of matches.
* I am unable to identify my relationship with matches as high as 122 cM at AncestryDNA and 23andMe, 141 cM at MyHeritage, and 192 cM at FTDNA.
* I have not yet found known relatives at LivingDNA, Geneanet, Geni, or Borland Genetics other than my uncle whose data I uploaded.
* I found this exercise very interesting. Compare my uncle at all sites to see how much the same person varies at different sites.
* I bet I’ve mentioned a few sites you didn’t realize you can upload your data to.

I thought it worthwhile to do my Uncle as well. He is my father’s brother, so he doesn’t share my maternal matches. Since he is a generation above me, he should average double the DNA that I do with my paternal matches, but of course he got different DNA than my father did, so that ratio should vary quite a bit for each individual match.

A) AncestryDNA – Was not tested

B) 23andMe – Was not tested

C) On MyHeritageDNA, My uncle has 21,131 matches (17% more than me). 
His top 10 matches are:

* Me, nephew, 1,994 cM, in tree
* #2 to #8 are all unknown cousins not in my tree with cM:
177, 172, 136, 133, 132, 131, 130
* #9 – Second cousin, 1x removed, 129 cM, in tree
who is my second cousin, 2x removed, 79 cM with me, not in my top 10.
* #10 – Third cousin, 121 cM, in tree
who is my third cousin, 1x removed, is not in my matchlist at MyHeritage.

D) On FamilyTreeDNA, My uncle has 30,284 matches (18% more than me).
His top match is:
* Me, nephew, 1,861 cM, in tree

His next 9 matches are all unknown cousins not in my tree with cM: 
180, 176, 173, 172, 170, 169, 167, 166, 166 

E) On LivingDNA, I logged in as my uncle, and it gave me a screen saying this: 

This was strange because long ago I uploaded his DNA, and in fact, my highest match is with my Uncle. So they do have him. The above screen would not let me go any further, so I could not check his results.

F) On GEDmatch, using Tier 1 One-to-Many, my uncle has 59,821 matches that are 7 cM or larger (0.7% more than me). His top match is:
* Me, nephew, 1,918 cM, in tree

His next 9 matches are all unknown cousins who are not in my tree with cM:
165, 136, 134, 132, 132, 130, 129, 129, 128.


G) On Geneanet, my uncle has 738 matches (3% fewer than me). His top match is me with 1,925 cM.  His next 9 matches are all unknown cousins with cM:
74, 57, 55, 54, 48, 46, 46, 46, 46

H) On Geni, I did not upload my Uncle’s DNA there.


I) At Borland Genetics, my uncle has 257 matches (70% less than me).
His top 5 matches are:

* his brother (my father), 1890 cM, in tree
* his nephew (me), 1805 cM, in tree
* his father (my grandfather), 1278 cM, in tree
* his mother (my grandmother), 99 cM, in tree
* his grandfather (my great-grandfather), 71 cM, in tree

The next 5 matches are unknown cousins who are not in my tree with cM:
64, 63, 54, 45, 45

Note that my father, grandfather, grandmother and great-grandfather’s DNA were partly assembled from my and my uncle’s DNA using the Borland Genetics tools.