loganom
Has there been any progress made on this IA? I was wondering if the Perl Web::Scraper would be appropriate for a spice. I saw yegg mentioned that an XML API might exist from history.com but I have not been able to turn anything up (hence my use of Web::Scraper). Anyway, I am currently working on this, as it seems to me that this has not received a lot of attention from developers over past months.
posted by loganom Community Leader4 years and 8 months ago Link

chaitanyanettem
Hey loganom. I had started some work on this earlier but I couldn't continue because of other obligations. I came back to this today (coincidentally) and wrote some code for scraping data from wikipedia. My piece of code is at https://gist.github.com/chaitanyanettem/...

Currently my code access all 366 days of the year and writes the events, births and deaths to files (1 for each day). There are a few bugs that I am trying to iron out.
posted by chaitanyanettem 4 years and 8 months ago Link
loganom
I'll get my code on git or other site so you can check it out later today, and I'll post the link here, but I've scraped history.com and find the fact that they have a lead story and a minimal number of other facts to be advantagous. How would you decide what to display?
posted by loganom Community Leader4 years and 8 months ago Link
loganom
posted by loganom Community Leader4 years and 8 months ago Link
chaitanyanettem
The problem with History.com is that there layout can easily change. Wikipedia on the other hand strives to maintain the same layout and changes are few and far.
posted by chaitanyanettem 4 years and 8 months ago Link
loganom
Yes, that is a fair point.

With wikipedia there is less for us to maintain, though it is harder for us to determine what story to return.
With History.com, as you pointed out, the format of the source could change more frequently. However, we do not need an effective heuristic for returning stories. They list only ~20.

I haven't been here very long. What do you all think is the most favorable choice?

@chaitanyanettem
@zac
@yegg
posted by loganom Community Leader4 years and 8 months ago Link
chaitanyanettem
I think for a start yegg's suggestion is workable. I can get that functionality done in little time.

If someone could clarify whether this should in fact be longhead and not spice then I will go ahead and get the necessary functionality up.
posted by chaitanyanettem 4 years and 8 months ago Link