Hacker News new | past | comments | ask | show | jobs | submit login

Sounds like you can use a macro on the OS level or a plugin like greasemonkey/tampermonkey.



Never tried that. I’m basically trying to setup a cron to login to this site every day and download a CSV.


Any of these would work fine for your idea and not be very difficult to do. (Assuming the site doesn't have any anti-bot measures). You'll need to create a cron that runs the automation script or opens a browser with your greasemonkey script installed.

Sometimes you can even do these with pure curl. POST the login form, get back the necessary cookie or token, then request the download URL, if it's easily predictable.


In my worst case scenario doing something similar, I had to go through the login process via Selenium, then download the file (it actually was a CSV for me too) in Python, by grepping the source code (Xpath and CSS selectors were useless) with a regex. There are ways to share cookies between Firefox and Python, and you could probably save a step by running Selenium from Python. Then make an HTTP request with the proper cookies, UA, and accept-* headers.


Nordpool energy prices perhaps?




Consider applying for YC's Spring batch! Applications are open till Feb 11.

Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: