Hacker News Reader

Comment by jefftk | Hacker News Reader

I have a group I still care about. Here's how I'm archiving it:

    git clone git@github.com:andrewferguson/YahooGroups-Archiver.git
    python archive_group.py <group-name>

See https://github.com/andrewferguson/YahooGroups-Archiver and https://www.archiveteam.org/index.php?title=Yahoo!_Groups

View on HN

I'm using https://github.com/IgnoredAmbience/yahoo-group-archiver , it has support for databases and files which is important for the groups I want to archive.

It's not my project, but I'm working on improving it, somewhat fixed the database support yesterday. Trying to figure out how to add support for links, but I'm stuck at link subdirectories/folders; I can't find a way to list the links that aren't in the root dir with the API (though once I have the link name I can get them with links?filename=DIR/LINK.url)

Does that script even work for you? When I run it, I'm prompted for my Yahoo! password, but after typing it in the program just spits out a "Login failed" message. The problem has been reported on the issue tracker but there's been no activity on it since February.

You need to add some parameters for the Yahoo authentication cookie values. -tc and -ty should be set to the cookie values from your browser, after logging into Yahoo.

It also needs some try/except handling to be added for all the cases where Yahoo fails to provide the requested asset.

Once you sort those things out, it works pretty well.

Could you please explain what this means and how to do it? This is not documented on the project page, and I'm having the same issue as the poster above.

login.yahoo.com creates 2 cookies. The program's help identifies two flags called -ct and -cy which I assume you're referring to. Am I to understand that to properly run the program, the command should be entered as so:

./yahoo.py -ct %cookieOneContent -cy %cookieTwoContent -u username -p password groupname

I don't know which cookie goes where so I tried it both ways, and both with and without the -u username and -p password flags; none of them worked, only returning HTTP Errors.

There should be at least two cookies, one called T and the other Y, on yahoo.com, the content of these looks like a long query string, "z=something&a=something&etc". Use T in -ct and Y in -cy, like -ct "z=something&etc" -cy "v=something&etc"

Sorry this is so tricky.

Ah, I see. It wasn't under login.yahoo.com then; the cookies I was looking for were under "yahoo.com locally stored data." This worked. Thank you.

e: actually, no dice. Got it to work on one group and then never again. A couple of the groups I've tried can retrieve messages, but never files or attachments. It always gives an exception when it gets to the files.

I'm offering a service that will extract all the yahoo group data from both private/public yahoo groups data for a fee.

My service: Includes - Public groups backup - Private groups extraction (will require admin credentials for the account) - Extraction formats include excel, csv and html

Not Included in the backup - Importing to new platform - Data Translation/Data cleaning - Additional post backup requests

Total cost: - Full backup $500

Payment details: - $250 for the first installment. - $250 after the last sample excel file is sent

After the first payment I will start work. Once I have completed the data backup, I will send a excel file with 10% of the final backup file as verification of my work. I will send the file once the final $250 payment is complete.

If your interested, please contact me at yahoogroupsbackup@gmail.com

- oran c

-u and -p don't seem to be needed anymore? The script complained so I dropped them. Also, I used the "F" cookie for -ce.

So: ./yahoo.py -ct "T cookie"-cy "Y cookie" -ce "F cookie" "group name"

Noob question: how to convert the downloaded json file into a readable message?

First run those commands, then run

    python make_Yearly_Text_Archive_html.py <group-name>

Happen to have a screenshot of the output of that? Just from skimming the source, I would imagine that messages in longer threads get pretty unreadable.

https://i.imgur.com/s7IiHO6.png https://i.imgur.com/V6QzKdT.png Run these scripts with python2, as it uses print statements. Also

   pip install natsort

They meant a screenshot of the output file.

trying this alternative: https://github.com/csaftoiu/yahoo-groups-backup

I tried this long time back. It requires a mongodb instance. I was not happy about it.

Doesn't work for me. The file scraper crashes with the default Firefox Selenium driver, and with the Chrome driver it immediately quits, falsely reporting that it successfully downloaded all the files.

If you don't use git authentication and use https, this works better:

    git clone https://github.com/andrewferguson/YahooGroups-Archiver.git

i'm getting this:

 YahooGroups-Archiver  master  python archive_group.py tns98

Archiving group 'tns98', mode: update , on Wed Oct 16 20:01:04 2019 Traceback (most recent call last): File "archive_group.py", line 143, in <module> archive_group(sys.argv[1]) File "archive_group.py", line 67, in archive_group max = group_messages_max(groupName) File "archive_group.py", line 89, in group_messages_max return pageJson["ygData"]["totalRecords"] UnboundLocalError: local variable 'pageJson' referenced before assignment

but nobody else seems to be facing this issue?

I got this error when trying to archive a private group, see the README for how to archive private groups.

ah thanks, wish that was documented. thought it was public.

tns98 is a private group. " To archive a private group using this tool, login to a Yahoo account that has access to the private groups, then extract the data from the cookies Y and T from the domain yahoo.com . Paste this data into the appropriate variables (cookie_Y and cookie_T) at the top of this script, and run the script again."

that was it, cheers. didnt realize it was private.

Reply on news.ycombinator.com