Importing Two Ged Files - Categorized in: Feature Suggestions

Now you've reached into something that I feel very strongly about.

I am and always have been dead set against merging other people's data into your own. I am shocked at how much importance is seen in being able to do this. This has caused a horrible problem: a proliferation of "junk genealogy" where little is proven or even questioned. The important thing seems to be adding 2000 more people to your family, whether or not they are truly connected - and then no work is done to verify that anything in that data is true.

But even if we forget about that, then there is the second problem: Let's say you get a program that has amazing side-by-side merge capabilities. It only takes 1 minute to figure out what you want to merge about each person. The 500 people you need to merge only take you 10 hours. Then, 2 months later you get an updated version of the file you had merged. Do you now want to go through it all again to get the updates? When doing this, will you be able to tell what was from your original data and what was from the earlier version of the file and the decisions you made about which data is better and why?

It's all wrong and a waste of time, I say. Instead, I think it is important to keep separate files that are received from different people or places. It is this concept that I am building into Behold. Behold can merge multiple files and you can select the families from each file that you want to display. The information about which files and which families can be saved in a Behold file. By opening the Behold file, it will reload all the files and families as you have specified them. That way all files can remain separate, but useable together.

If you want to then save this as one GEDCOM file, Behold will allow it (in Version 1.5). But I wouldn't recommend doing that.

Now there are two related ideas. One is to be able to identify which people in different files are the same. Then the data can be displayed together in the Everything Report. Doing that will allow the direct comparison that you are interested in, but will alleviate the burden and problems of merging. I want to allow the identification of identical people to be stored in the Behold file so that this info can retained and reloaded when needed. Hmmm. It's not in my Future Plans, so I'll add it.

The other idea is to simply use Behold to load multiple files and look for possible people matches. I've had that concept for a long time, and have some ideas on how to do it, but the details need to be worked out. Possibly color highlighting or something like that.

I've rambled on a bit here and I'm not sure if I completely addressed your suggestion. Assuming you have a good tool that allows you to look at the data combined from separate files, do you still have a real need to merge them that I may be overlooking?

Louis

Whew. You had me worried. First about merging for the sake of merging, and second about wanting a refund ... also just kidding.

My problem with side-by-side merge/display programs is that they need to have selected fields to compare: Name, Birth date, Birth place, Wife, Children, etc. There is a lot they can't show, such as notes, source references, multiple names, all the pictures, etc. In other words, they pick and choose what they show and cannot show everything. They do this in yet another modal box that takes over your screen. That is not how Behold is set up to work.

Behold's comparison would be to take the "Everything" about one person in one file and displaying it with the Everything about that same person in another file. The data will not be physically merged but will display together, which is I guess what you meant all along.


I could get Behold to put all the data from the first file just before the data from the 2nd file, e.g.

F1-14
John Smith
b. 14 Nov 1900 in New York, NY. [S23]; 17 Nov 1900 in Long Island, NY. [S42][S16-3][N5]

F2-42
John H. Smith
b. Approx 1902. Estimated from marriage date.


Or I could get Behold to display them with each specific type of info shown together, if I can figure a way to do it without it being too messy, e.g.:

F1-14; F2-42
John Smith; John H. Smith
b. 14 Nov 1900 in New York, NY. [S23]; 17 Nov 1900 in Long Island, NY. [S42][S16-3][N5]; Approx 1902. Estimated from marriage date.


I haven't put too much thought into this and nothing is final yet, but your comments are helping me to evolve this idea into something that will work well.

The goal, of course, is to have a format to make all the info available so as to help solve those nagging mysteries and inconsistencies. Then your conclusions and best guesses can finally be properly documented.

Louis

Despite my big disclaimer in my first message: Yes, I see that I must add this capability to Behold. I agree that there is a need for such a function for many valid reasons.

I'm not sure how I will do it yet. Maybe by simply selecting one or more people then using a right mouse menu function (or other ways) to get Behold to combine them. Items from the 2nd (or 3rd or 4th) files will be tagged with a Source that will indicate what file it originally came from.

(I'm using the word "combine", but yes, it is a merge).

But it will, like most things in Behold, be very different than other programs. Because all the info is visible in the Everything report, seeing it, selecting it, and using this merge function will be unbelievably easy to do. It will combine notes, sources, links and not miss anything. The merged data will look identical to the unmerged data (with the change of some reference numbers and addition of source references). Then manual edits (which will also be incredibly easy to do) can polish it off.

It is because this can be made so easy to do that I am worried.

Louis

Regarding ranting about junk genealogies, this is interesting:
randomgenealogy.com/2005/08/13/lost-fortunes-in-the-family-tree-slight-rant/

Further thinking on this: "Linking individuals from different databases ... has to be the way of the future".

See my thoughts in My Behold Blog posting for Wednesday, November 23, 2005.

Louis

Thank you David for your ideas.

I think your advice is the correct advice. I have very strong feelings AGAINST merging other people's data into one's own. I'd go even farther and say that you should NOT add other people's data to your own.

Behold will be introducing a concept I call "Virtual Merging". I discuss it in my Behold Blog post of August 28, 2006.

My idea is that (with Behold's help), you specify which people in the files are the same, and Behold will produce one report for you, merging the information for those identical people from the files.

Does that make sense to you?

Louis

Seecan:

Currently, Behold will import two (or more) GEDCOMs and display them together. The current weakness is that it currently has no way for you to indicate the people who you know are the same in the two families, so every duplicated person will be shown twice.

In the case of your paternal grandfather and paternal grandmother, it is likely that your parents and all their descendants will be in both files, as well as possibly some cousins who may have married between the families and their descendants.

That will be fixed when I implement Virtual Merging, which will give you ways to simply mark the people that are the same so that their data and their descendents' data can be displayed combined by person. Currently, that is on my future plans page, but after Version 2.0 and editing. Although every time someone asks about something, like you just did, it moves up in my internal priority list.

In the meantime, you should still find the current merging very useful. The people will be shown separately, but the Name Index, Place Details, Source Details and everything else should prove very useful and allow you to find data problems in both your files.

Hmmm. Now you're getting me thinking of a way I can implement Virtual Merging much more quickly. I can on the Organize Families page, allow you to add the person from the Second GEDCOM file, and allow you to specify that they are the same as a previous person on that page. You'll only need to do that for a few of them, being the beginning of the common lines. It should work out quite well. I may try to do that prior to going on. Thanks for the push.

Your second question is a very good one and I'm happy to answer. Although no one can predict the future and I may get run over by a bus tomorrow. But barring something like that, I'm planning to continue work on Behold for the foreseeable future which is likely a minimum of 10 years. I'm absolutely positive it will be successful because there is no other program like it, and it's the program I will want to use to record my own genealogy (which will extensively test it as well as I start to enter my masses of data).

Currently, I have nothing in place should that bus be lurking. But I have been thinking about it. If I grow large enough, I'll hire others to help me, including programmers. (I'd love to get big enough to sponsor my own Genealogy Cruise!) If not, and Behold remains of interest, I may sell it to someone willing to keep it going, or donate it to the Open Source effort. But Behold's my baby. I won't let it die.

Besides, $40 is less than you'll spend on a single upgrade of another program. So after 10 years of use, I'm sure you won't be lamenting the loss of the $40. What will be more important to you will be if you will not lose all your hours of work and be able to continue your research with another program. Make sure your program will export compliant GEDCOM. Behold will. Many programs don't

Louis

Well, my idea for Virtual Merging is extremely necessary. Too often, people are merging other people's GEDCOMs into their own, trusting that other people did theirs correctly and proliferating what are called "junk genealogies".

Not only that, you WANT to keep your work separate. If relatives send something to you, why go through days or weeks of work to merge theirs into yours, only to have them send you an updated GEDCOM six months later. That time could be much better expended doing your own research, rather than on some mechanical merging that could be done virtually.

I'll make the virtual merging painless. The most difficult thing for me will be figuring out how to help identify who might be potential merge candidates, since names and places might be spelled a little differently, dates might be different, and spouses and children can be different or missing. Once I figure out something for that, the rest is a piece of cake.

Louis

Thanks Dave for the review of GenP - which I actually haven't tried myself. Unlike GenP, Behold maintains the two-way relationships. When one way is created, the other is as well (and Behold will identify one-ways in the GEDCOM that should be two-ways).

Hopefully, when I add virtual merging to Behold, I'll be able to make the identification of identical people very simple, so that it shouldn't take more than a couple of clicks. Then, somehow it will have to cascade to spouses, children, parents etc and allow some sort of checklist confirmation of the ones to combine. But that will be worked out when the time comes.

Louis

Theresa,

I doubt I will be implementing a side-by-side feature. My current thoughts are that I'll probably be mixing the data together, with events for each person ordered by date, and allowing you to highlight the data from the different GEDCOMs with a different background color.

You would not want to use any genealogy program's merge utility to merge people in different censuses at one place. All program have people-based merges, that don't work for sources. You need a source-based data entry system, one that will suggest likely people that the source refers to. There isn't one out there that I know of (yet).

Louis