I’ve been experimenting with AI tools as a way to more efficiently create a database for an 18th-century census I’m working on. To be clear, I’m not asking it to transcribe from the originals (as above) as some scholars are now experimenting with. But instead working from the transcription Tom Peace, Renée Girard and I developed for our Digital Disruptions repository of documents.
I tried several tools and by far the best results came with Anthropic (claude.ai). The basic model of Anthropic doesn’t allow sharing of links, so next best thing:
After some discussion with Claude, it was able to generate a table that looked like this, which is pretty close to what I’m looking for. There are 23 headings and I can imagine two or three more, but for now this get at all the consistently offered data (names, ages etc) and columns for comments on improvement and “additional notes” [uncategorisable stuff].
This is very good, but scalability became an immediate issue. Organising the data for five households was manageable, but Claude made clear it could not handle significantly larger chunks of data – even if I were to upgrade to a “pro” version of the service. We then had a slightly confusing conversation, but arrived at this:
However, you’re correct that there are usually limits on the amount of text that can be input in a single message. A subscription might indeed allow for longer input limits, which could allow you to paste more text from your data file into the chat at once.
To work with your larger dataset, you have a few options:
- If you have a subscription, you may be able to paste larger portions of your data into the chat at once.
- Without a subscription, you can still work with larger datasets by pasting portions of the data over multiple messages. I can help you organize each portion as we go.
- You could also summarize or describe larger portions of the data, and we can discuss how to structure it based on your description.
I apologize again for the confusion in my previous response. Thank you for bringing this to my attention. How would you like to proceed with your larger dataset?
So, (a) getting a subscription would allow me to enter larger amounts of data (though not much bigger), and (b) Claude can offer suggestions on how to stitch the pieces of output together together – I’m not sure I get option 3.
