Does it make sense? Bots rarely make HTTP requests for images, css, video clips, large JS files, custom fonts, etc. Real people do. A well-written bot just seeking some specific data can often complete it's task with less than 1% of the resources that would be sent to a "real" user.
I am not sure what "bots" you are talking about here. When I wrote a web scraper to get my Air Canada point values I used a script that fetches the web page and parses it. It was the only way i could get it to work. I had to steel the session token from the browser cookie in order to make it auth
I guarantee that, with some effort, you could write a script to emulate every HTTP call -- logging in, accepting the cookie (just a value in a Set-Cookie header), and requesting the point values, making sure that cookie value is in your Cookie header. Just because you could "only get it to work" one way, does not mean there isn't a far more efficient way.
Dude, this is my story about my experience. The context provided above is relevant to what I was doing. This has nothing to do with theoretical possibilities that you are speaking about it. You are in the wrong thread.
Those are all static files that are easily (and typically) cached in front of the application. Pulling customer-specific data from an authenticated session taxes the application (and DB) directly.
I think you are over-estimating the use of caching in a lot of industries and a lot of companies. Further, a company that aggressively caches static files should also recognize the benefits of caching their most common database queries. Another replier to my comment mentioned his Air Canada point totals. The original post is about American Airlines. Point balances change infrequently. An airline could easily query every active customer (had a point balance change within the past 6 months) every 6 hours and keep all those values in memory, dramatically reducing individual DB queries. Or not, and instead choose to sue a very popular blogger and builder of a tool used by your best customers, pissing everybody off and looking extremely petty and customer-unfriendly in the process.
True, although the more you lock down, obfuscate, and hide your data, the more the bot-writer is going to use the heavy guns to penetrate you. Points Guy, and Award Wallet are both attempting to provide a service that people, especially American Airlines's most valuable customers, obviously want -- AA could easily work with them, instead of against them (or provide the same service themselves).
