Here are some links:

http://www.igvita.com/2008/02/11/nginx-and-memcached-a-400-b...

http://lserinol.blogspot.com/2009/03/speeding-up-your-nginx-...

I'd think about switching back to a public interface that has been through these battles before. You could always keep your server up and use Nginx or HAProxy to front it for you and just pass the requests on.

Good luck with your app!

Fork Bombs are easier to pull off from the command line. Not so much with DOS. I would say that loading a large webserver for each request would use up more resources than a tiny C program. Thats the whole point of the post.

I would use Go for this service myself. Defending against abuse is hard whichever way one goes.

PHP was the culprit here, but I can't help but think you'd have had the same problem if you were trying to do the same thing with RoR, any Java app server, or any of the Python frameworks.

Likewise, node.js or Twisted probably would have been an equally effective replacement.

Right now mostly working on CherryPy's WSGI server.

Using AJAX for this would give you MUCH less accuracy than plain-old JavaScript with time delta of user time to server time:

If you just need to time sync - you could receive server time once (remembering user time, when you sent request for server time), then just compensate user time with that value.

In PSEUDO-JavaScript (client-side):

  // do this once:
  var user_time = (new Date()).getTime();
  ajax.call('/server-time-in-sec-since-epoch', 
    callback: 
      delta_time = recvd_server_time - user_time;
  )
  
  // then at any given second real server time is:
  var current_server_time = (new Date()).getTime() + delta_time;
  // no need for ajax calls

You could even incorporate the timer into a single request.

    do {
        sleep(1);
        $seconds_remaining = fetch_auction_time_from_memcache();
        echo "<script>updateActionTime($seconds_remaining);</script>";
        flush();
    } while ($seconds_remaining > 0);

Longer running connections are a little more problematic, but some client side code should be able to handle the connection being closed. You just need to make sure your web server can handle the number of connections your are expecting. Apache is particularly bad for this. Something like nginx should perform better.

Replacing PHP/Apache fork/threads with a C daemon is a good migration, though most any language with an async sockets library worth a damn should be able to handle thousands of simple requests every second.

Don't get me wrong, it's an amazing racket; Wavee makes 75 cents for every bid coming in to increment the value by 1 cent. Saw a $150 iPad. That iPad has already earned Wavee's owners $11,250 without even being sold! And the use of Facebook to gain the trust of people is a good marketing tactic, but it doesn't skirt the fact that creating fake Facebook users with a bunch of friends is easy (get some pictures, feed some content in from any one of the millions of open twitter accounts, etc.), or that you have a real incentive to perpetrate fraud.

Really though, being the most honest crook among crooks still leaves you being a crook. That's why some countries have outlawed this particular kind of scam.

I honestly didn't know (and still don't) if you can make any money doing an honest penny auction site. We are just breaking even with this one so far.

The fact is that every time an auction goes up we have more risk than anyone else involved. If I put up a $500 iPad it could go for extremely cheap. The lowest one has ever gone for is 64 cents. That means we made less than $64 and still had to buy and ship that $500 iPad. And we have never sold anything for more than $26 which still isn't the $2600 you think it is because some packages give you more bids per dollar.

The legitimate complaint is when sites cheat. Which I would guess alot of them do. Using Facebook is not some trick to get people to trust us. Its a way for people to verify for themselves that they are in fact bidding against other real people.

How can you say that its a scam? Do you think we are just out there creating all these fake accounts with fake profile and personal photos and relationships etc. I can see maybe if we had 5 or 6 or even 20 users. But we have 100's of winners. That's a stretch even for the most paranoid and cynical of people.

Not everybody wins every time. But the data we have so far says that the majority of users that buy more than just a few bids are actually the ones getting all the deals. Its the people who come in and spend $24 expecting to win a $1500 item then leave when they don't. Those are the people who are losing. The users who are logical enough to see how the system works are the ones who get far more than they put in.

And some countries outlaw all kinds of crazy things, some countries are considering passing legislation to ban homosexuality so I guess we should all agree that its bad now too according to your theory?

Let's give you the benefit of the doubt for a moment that you are actually honest. We'll say that you aren't running bots, fake identities, etc. That's fine. My basis in calling you (and those that run businesses like yours) a crook is not founded on that (though I have no doubt that other companies are doing as much, if only to boost the value and bidding on an item, but I digress).

Try to remember that the fundamental operating principle of your business being profitable is your selling the vast majority of your customers absolutely nothing. They aren't getting a good or service for any bid that doesn't win (which still costs them money). They get nothing.

The money to purchase those items must come from somewhere. If you are breaking even (as you say you are), it's not coming from the people who are "winning", it's coming from the people who are losing. If/when you are making a profit, it's not because the "winners" are necessarily paying that much more for an item (they won't have bid enough plus paying for the total value to pay for the item itself), it's because you've got more losers who are putting money into something without getting anything in return.

Your business is breaking even, and may eventually be profitable because of all of those who come in buying $24 worth of bids, failing, and leaving. Any business that requires it's customers be ignorant in order to make money is fundamentally a scam.

Also, your conflating countries making homosexuality illegal with countries making illegal a business that bases it's operations on exploiting the ignorant, is a fundamentally flawed argument. One is based on basic human rights to be who they are and behave in ways that causes no harm to others. The other is one that profits from people who don't know any better. One is a human rights travesty, the other is the outlawing of an enterprise with margins that organized crime wishes they could have (in the case of a "successful" site like Wavee and others). Trying to claim their equivalence, or that based on "my reasoning" they are equivalent, is dishonest, and really, troll-like behavior. That may fly in some forums, but it doesn't fly here. Try again.

Your business model requires minimizing the number of those people and maximizing the number of suckers. There's a reason they're seen as scams and exploitative.

For instance, consider what David Heinemeier Hansson says about Campfire, the chat software he helped developed. First written in Ruby On Rails, it soon became clear that the code that polls to see who is in the chat room needed to be as fast as possible:

"We rewrote the 100 lines of Ruby that handled the poll action in 300 lines of C. Jamis Buck did that in a couple of hours. Now each poll just does two super cheap db calls and polling is no longer a bottleneck. Campfire and a shared todo list is different because they’re not working on a shared resource. There’s no concept of locking. Or two people dragging the same item. So a 3 second delay between posting and showing up doesn’t matter. It does when you’re working on a shared resource."

http://www.ruby-forum.com/topic/62907

Later they tore out the C code and re-wrote it in Erlang.

Running Erlang has some overhead, sure, but that's because it's designed for distributed systems where you can pull a plug out of the wall without interrupting service. I wouldn't use Erlang for number crunching, but using it as a glue language for a networked system hits all its strong points.

I know it's splitting hairs whether bytecode + a VM counts as compiled or interpreted (it's both, really), but compiling to bytecode rather than a pure interpreter usually makes enough of a difference performance-wise that it's worth giving some credit.

it doesn't make sense to make a new connection every 1 second, esp. to apache. that is where your prob was, not php

The world is built from bailing wire and duct-tape. There are probably a million better, smarter, less technology illiterate ways to solve this problem, but that really doesn't matter. What matters is getting out there and doing it. Being able to do this sort of app 'correctly' would be an edge, but only to the person who can do it 'correctly', and is out there doing it.

I wish you the best of luck with your duct-tape!

You always have to use the right tool for the job. It requires a deep understanding of what is actually going on inside the server when you write a line of code. PHP doesn't magically absolve you of that.

It really has nothing to do with PHP, C, Ruby or [insert your most reviled/loved technology here]. Calling a complex runtime for hundreds of near-contentless requests per second on a single machine is a really bad idea, no matter what environment you use.

Also, I'm sorry if I snipe from the cheap seat here, but 1 request per second per user doesn't seem like a great solution to your problem either. It might be more appropriate to just leave the HTTP connection open and push new data out through that when it becomes available, e.g. when something about the bidding process changes.

FastCGI is worth looking into - a lot of popular webservers support it, and it's less of a complete model change than switching to an event-based system (e.g. node.js) or an MVC framework.

* Or SCGI, which is a newer, simpler design with similar goals.

http://jehiah.cz/a/ntp-for-javascript

Then have a dedicated ntpd daemon running. That will be way more extensible, maintainable, and scalability than a custom C program.

as much as i dislike like it, i doubt the problem has much of anything to do with php. a simple fastcgi server hooked into lighttpd would have probably had the same outcome of better performance, or even apache with mod_php.

A nice discussion on why PHP DOESN'T suck: http://stackoverflow.com/questions/309300/defend-php-convinc...

It's that it exists in the same universe with Python, Ruby, Java, Clojure, Erlang, etc.

If you have the freedom to choose your implementation language, and those are your alternatives (to name just a few), that's when PHP is harder to justify. Not that it's terrible in isolation. In relative terms.

stringstream response; response << "HTTP/1.1 200 OK\r\n" << "Server: BTB Auction Updater 1.0\r\n" << "X-Powered-By:BTB Update Engine\r\n" << "Content-Length: " << msg.length() << "\r\n" << "Content-Type: text/html\r\n\r\n" << msg;

  int n = write(s, response.str().c_str(), response.str().length());
  if(n < 0) error("Error writing to socket");

Right now the C app listens on a different port that lighttpd will redirect to if a pertinent request comes in.

There is a parent process that will fork upon connection and pass the socket to child which pulls the requested data and quickly returns it.

The one thing you will also need if you go this route is a SIGALRM handler. You just basically call signal() and set a flag before each socket read and write. So if the socket takes too long to return the signal gets called and kills the child process. Then you clear the flag on the other side of the socket read/write call so that the signal doesn't kill the process if the call returns on time.

p.s. I'm part of the company whom made it.

If you have 100 users, each one waiting for a message in their queue, then you can "broadcast" a single message to all of the queues (or a group) with one command. All the users waiting for a message in their queue will get a copy.

Another approach which is hack-ish imo, is to use something like jabberd to broadcast messages.