Login to participate
  
Register   Lost ID/password?

Louis Kessler’s Behold Blog

DMT 4.3 - New FTDNA File Format and Matching Algorithm - Thu, 26 Aug 2021

I have released Version 4.3 of Double Match Triangulator which includes changes to allow it to read the new Family Tree DNA segment file format.

On July 1, Family Tree DNA announced changes to their matching algorithm along with other updates on their blog: Updates To Family Finder, Featuring Improved Matching And A Soon To Be Released Chromosome Painter.

On July 7, they sent an “Important Update” via email to their users that they were continuing to refine the matches and hoped to complete this by July 23.


Changes to FTDNA’s Matching Algorithm

Family Tree DNA, like the other DNA companies, considers the details of how they do their matching proprietary. All that really can be analyzed are how the resulting matches have changed.

The most important change is that now segment matches are only reported down to 6 cM. Previously Family Tree DNA reported all matches down to 1 cM. The company was often criticized in the past about reporting these small segments which were often random matches that were not IBD (Identical by Descent).

As a result of this change, the total cM reported between two people should be more compatible with the matches reported at other companies that cut off their matches at 7 cM.

But the change in the matching algorithm has also changed the individual matches. Here, for example, is a comparison of two 3rd cousins.

These are the segment matches Family Tree DNA now reports:

image

So Family Tree DNA now reports 9 matching segments totalling 95.1 cM.  The smallest segment is 6.1 cM and the largest is 18.7 cM.

Previously, these were the segment matches for the same two 3rd cousins:

image

Family Tree DNA previously reported 21 matching segments totaling 110.7 cM. The smallest segment was 1.7 cM. The largest was 18.1 cM.

For comparison with the new matches, let’s ignore the 15 matches under 6 cM (shaded in the above table). This leaves us with 6 matches totaling 63.3 cM with smallest 7.2 cM and largest 18.1 cM.

Those are quite different results. None of the Start Locations or End Locations or Centimorgans or Matching SNPs are the same between the old and new matches.

Looking at the 9 new segments individually:

Chr 2:  This segment is 6.1 cM new and 5.0 cM old

Chr 4:  This segment is 7.7 cM new and 9.9 cM old.

Chr 7:  This segment is 7.7 cM new and 4.7 cM old.

Chr 11 (1 of 2):  This segment is 10.1 cM new and 8.2 cM old.

Chr 11 (2 of 2):  This segment is 7.0 cM new and 7.2 cM old

Chr 18: This segment is 9.9 cM new and 9.7 cM old

Chr 20: This segment is 18.7 cM new and 18.1 cM old

Chr X (1 of 2): This segment is 14.0 cM new and 10.3 cM old

Chr X (2 of 2): This segment is 9.4 cM new and 3.8 cM old

2 of the 9 segments are smaller with the new matching than they were with the old matching.

7 of the 9 segments are larger with the new matching than they were with the old matching. And 3 of those old values were smaller than the new 6 cM threshold.

The good thing is that all of these 9 segment matches overlap. Even though their starts and ends and lengths don’t match exactly, you can tell which ones are the same. None are missing, at least in this example.

The largest cM difference is 5.6 cM and the largest start or end position difference is 3,176,824 base pairs, both of which are significant amounts.

What this does show is that you should try to avoid comparing segment matches done with the new algorithm to segment matches done with the old algorithm. In cases where you only have an old segment match file and cannot get the new match file for that person, then you may have no choice but to use the old one.


Changes to the Chromosome Browser Results File

Along with the other changes, one change Family Tree DNA did not advertise was a change to their segment match download, which they call their Chromosome Browser Results File.

One problem Family Tree DNA’s website had for over the last year and a half was that if you had a large number of matches, you couldn’t download your segment match file at all. Clicking on the “Download All Segments” button:

image

would result in a little progress circle going round and round and round … and it would never stop, even after hours of letting it run. The files that were being generated were just too large and the website’s servers would time out and stop processing without informing you.

So that had to be fixed, and it was. By only including the segment matches 6 cM or more, the files were already made much smaller and required less processing to produce. Also column 1 used to contain the name of the DNA tester and was the same in every row. So Family Tree DNA excluded that column from the file, making the file even smaller.

The final result:  Now even the largest files will process and complete, and after not more than a couple of minutes, the download file will be produced. And that is great news.

The bad news for Double Match Triangulator is because the DNA tester’s name is no longer in the Chromosome Browser Download file, there is no way for DMT to tell who the tester is. The tester name is not part of the downloaded file name either.

So users of DMT using the new Chromosome Browser Download files will have one more step to do. You’ll have to open the file and add into cell G1 the tester name as it is referred to in other files, and save that. Full instructions are in the DMT help file on the page with Family Tree DNA instructions.

My Week as the WikiTree Challenge Guest - Sat, 24 Jul 2021

@WikiTreers is a free One-World family tree set up like a wiki. It has been in operation since 2008. It currently has over 27 million profiles.

The community of people who use WikiTree set up various online social events and challenges amongst themselves to encourage improvement of the quality of their tree. They declared 2021 their “Year of Accuracy” and this year thought of a very innovative way to promote WikiTree and get new people involved. Each week, they invite a well-known (in the genealogy world) guest to add their ancestors up to their great-grandparents to WikiTree. The challenge part happens when the WikiTree community goes at it to extend the guest’s tree adding sourced profiles with documented biographies. They collaborate with each other in a Discord chat room. The challenge part is to see who can acquire the most points, by breaking brick walls (when compared to the guest’s main tree elsewhere) and adding profiles for new ancestors and their siblings, with additional credit given for any profile edited. The points really are just for fun and extra motivation to help do the best job possible with the guest’s tree.

The guest’s week starts on a Wednesday. There is a livestream with Mindy Silva, the WikiTree Challenge coordinator, the Team Leader for the week, the previous week’s guest and the current week’s guest.


My Intro, Wednesday July 14

My week started on July 14. It was the end of the John Boeren’s week. The first half of the livestream was what was called the “reveal”, where the team reveals what was done on the guest’s tree and who did the best, points-wise, in the challenge. John has all his roots in The Netherlands. The Wikitreers managed to add over 1000 profiles of John’s ancestors to WikiTree, just about completing his ancestry to 8 generations deep, in some cases breaking brick walls by extending John’s tree further than he had done in his own research.

This was followed by my introduction and a review of some of the brick walls that I have in my research:


(Note: It appears WikiTree removed the video so it is no longer available as of Jan 2022)

Unlike some of the guests, this was not my first exposure to WikiTree. I heard about the WikiTree challenge in January and that prompted me to join WikiTree in February. I then added my ancestors and started to write up their profiles on WikiTree.

I really enjoyed watching the Wednesday livestreams with the various guests, and seeing their reactions at their reveals. The WikiTreers worked very hard and always surprised the guest with what they found.

Then in June, they had Jarrett Ross, the GeneaVlogger as the guest. There was a tree I thought I could help with, so that week, I became one of the challengers working on Jarrett’s tree. It was an intense week of working on someone else’ genealogy, but was a lot of fun.


Wed July 14 to Wed July 21

With my involvement up the that point, I was hoping that Mindy would allow me to be one of the challengers, or at least help the participants in the Discord chat. Unfortunately, that was not allowed. I was not supposed to know anything about how they were doing. It was all supposed to be a surprise to me at the reveal.

So I followed the rules, stayed off WikiTree, and I even uninstalled Discord from my phone so I wouldn’t get notifications that might clue me in. I was very honorable in that way because I realized it would be more fun for me to be surprised at the end.

But I was allowed send one-way messages to the WikiTreers. Jarrett had done this two weeks earlier with a few tweets from his Twitter account. So I did the same and sent out a series of 21 hints via Twitter including the @WikiTreers address to make sure they’d see them. e.g.:

I had no idea what they were working on, but those hints I sent out made me feel like I was involved.

Every Saturday, WikiTree has a morning livecast with Sarah Callis, Mags Gaulden, and Mindy that summarizes the WikiTree community’s activities for the week, ending with a brief summary of how this week’s WikiTree Challenge is going. Mindy couldn’t make it this week and invited me to take her place. Of course, nothing of significance was going to be revealed to me. I was to be kept in the dark.


My Reveal, Wednesday July 21

I kept myself busy on Tuesday and Wednesday diverting my mind from thinking about WikiTree by working on a few projects, cleaning my desk, and adding sod to my lawn.

I knew my challenge would be tougher than the others. All my grandparents and only one set of great grandparents had emigrated to Canada from Russia or Romania in the early 1900s. That meant that any deeper research would require Russian or Romanian records. Those records do exist, but very few of them have been digitized, translated, indexed and been made available online. I have had success on several of my lines using specialized researchers who have access to the archives in Russian and Romania and have been able to find records for me. I knew the WikiTreers would not be able to do that.

To make matters worse, I had been adding profiles to my tree since February, so the WikiTreers didn’t even have very many new profiles they could add for me.

Sure enough, they realized they would not be able to take my genealogy further back, so they wisely did the next best thing and focused more heavily on the siblings of my ancestors that had come over and worked to improve their profiles, find new sources, newspaper articles and other valuable information that I had never seen before.

This was WikiTree’s summary of my week:
Highlights from Louis Kessler’s WikiTree Challenge - WikiTree G2G


Excellent Results

Although they may not have extended my genealogy, the results were extremely rewarding to me. I had a few brick walls, not ancestral, that I really thought they could help me with and they did.

My great grandmother’s brother was Louis Segal who married Sophie and had a daughter Sarah in Romania. They moved to England in the 1900s, went to Canada in the 1910s and ended up in Jacksonville, Florida in the 1930s and then I lost track of them. The WikiTreers found their ship record to Canada as well as the likely 1940 gravestone of Louis Segal in Jacksonville, plus other twists and turns and tidbits to that story. They added all that information to Louis’ profile on WikiTree along with sources. I’ve got enough there to solve most of my big mysteries about him. I’m currently working on rigorously going through his biography and forming conclusions and reorganizing it appropriately. The “Research Notes” section of WikiTree biographies is a wonderful tool for this type of analysis.

The WikiTreers also added valuable information like this to other relatives. It will take me several weeks (at least) to go through all the work they did and review the new information found.

They attempted to look at the Rothschild family for me. Although they could not connect my wife’s Russian-based ancestor to the family, they added a lot of Rothschild profiles to WikiTree which was valuable work.

Another absolute breakthrough happened when I was going through some of my documents to provide for my hints. My number one brick wall is my father’s stepfather Louis Kessler and his first wife Sarah Katkow. They were supposedly from Ogec, Russia according to the 1916 Canada Census, and I couldn’t nail down where that was. That is until I found two death records for Sarah.


I had always thought those were two copies of the same death certificate. But they were in fact death registrations for Sarah registered in two different municipalities. And the 2nd one lists Sarah and her parents’ place of birth being Odessa, Russia!  The mystery of Ogec is finally solved. It was nice for me to contribute significantly to my own research during this challenge week.


Mondays with Myrt – July 26

I discussed my week on the WikiTree Challenge on Mondays With Myrt (Pat Richley-Erickson) starting about 13:50 in until about 47:00:


Going Forward

I know how long it takes to write good sourced biographies. It involves cross referencing all the information you have and putting it together into something that makes sense.

So the biographies that were added during my WikiTree Challenge week are very much appreciated. I’ll be maintaining those into the future and using WikiTree as the place I write my ancestors’ history.

Great thanks to all the WikiTree people who worked so hard on my tree this week.

There are still 5 more months of the WikiTree Challenge. Great guests are lined up. There are a few coming up, like Drew Smith and Melanie McComb, whose trees I plan to help with. Looking forward to the fun.

Come see me. This is my week on the WikiTree Challenge - Mon, 12 Jul 2021

This Wednesday, July 14 at 3 p.m EDT, I’ll be introduced as the @WikiTreers guest for the week.

I’ll be on the panel of the Livecast which will be about an hour long. The first half hour will be the “reveal” of John Boeren, whose week they’ll have just finished up. The next half hour will introduce me and the challenge that the WikiTreers will have with my tree.

The July 14 livecast link is: https://www.youtube.com/watch?v=zoHzM2AMRwg

500px-WikiTree_Image_Library-260.png

The livecast of my “reveal” next week will take place Wed Jul 21 at 8 p.m. EDT on both WikiTree’s You Tube Channel and WikiTree’s Facebook Page.

Note the intro is at 3 p.m. EDT and the reveal week is at 8 p.m. EDT. The first week was moved earlier to accommodate John Boeren who lives in The Netherlands.

Come to the livecasts so you can add comments and chat. If you can’t make the livecasts, both sessions are recorded and will be available afterwards via WikiTree on their YouTube or Facebook pages.


What’s Going to Happen

I have seeded WikiTree with my family research back to about great-grandparents, which to be honest, is almost all I know.

Genealogists who use WikiTree (known as WikiTreers) have been participating in this challenge since the beginning of the year. Every week they have a new guest and spend the whole week working on the guest’s genealogy. They try to break brick walls and find new ancestors, and add their nuclear families (i.e. siblings and children). Along the way, they check all the information and add sources to back up every fact. This is part of WikiTree’s “Year of Accuracy” and that is what they are striving for.

They also improve each relatives’ biography and try to find interesting details of their lives that will add some life to that person. Biographies are possibly the most valuable contribution you can give your ancestors.

The work they have done for their other guests so far have been fantastic. For example, watch Devon Noel Lee’s reaction and Connie Knox’ reaction to what was found for them when they were the guest.

My genealogy has always been an interesting but difficult challenge. All the 10 “grandparents” I research (my 4, my wife’s 4, my father’s stepfather and 1st wife) immigrated to Manitoba or Saskatchewan from Russia or Romania in the early 1900’s. Russian and Romanian records are not easy to find and not easy to translate once found. It will be a real challenge.

But my experience with the WikiTreers for the past 6 months during the challenge is that they have all shown to be excellent researchers who have been gaining expertise as the weeks go by. They share what they’re working on with each other on the chat that the challenge uses at discord.com. And more often than not, they figure out a solution to most problems. And they all had fun doing it as well. I tried it for a week when Jarrett Ross was the guest and it was thrilling.

Whether or not they break 100 brick walls, or no brick walls, I know they will be improving all my profiles at WikiTree, adding sources and research notes, and providing many new paths for me to explore with my research. I know I will be very surprised at the reveal, likely multiple times.


Radio Silence

I had hoped I’d be able to help as a participant in my own challenge, and do research with all the others. But that’s not the way it works.

So I am respecting the rules and as of my intro on Wednesday, I will stay completely off WikiTree and not peek one iota. I will not go on the Discord chat channels, and I’ll even uninstall Discord from my phone to prevent notifications from it.

I’ve got a couple of projects to keep me busy during the week. But I can tell you right now: it’s going to be a looooooong week!

So please join me if you can, at my kickoff and at my reveal the week after.