Well, it's not successful right now but it could be in some months:
Two years ago i've started working on an algorithm to find the perfect time for social media posts on facebook & twitter to improve user interactions & engangement. Right now it works so good for me that i've decided to extend the project by building an (invitation-only) SaaS Startup to get a proof of concept: https://realtarget.com
Because it's just a project beside my main job my girlfriend (project manager) and best friend (senior full stack developer) support me to lighten the workload.
If you like, i'll post a note on HN when the alpha version is ready.
A friend of mine is a data analyst at a large market research company and analyzes more than 10 billion cookies a month using Hadoop from apache:
http://hadoop.apache.org/
well let's say you want to create a minute by minute report on URL's visited in an ecommerce site that has a large amount of usage. Every minute I want to process all those URLs and decide if I want to create a flash sale on the site. Since this was an example I just thought of, let's imagine that there are 400k URLs visited every minute.
That would be a decent definition of what I meant by large amounts of data. Sorry for not making that clearer before.
Your friend's example is pretty good. Curious how they work on the data in memory.
It depends on how many unique URLs you have. If you have like a few hundreds or even thousand I would just use a simple hash table with counters to see what's a hot sale right now (I guess that's what you want in your example). If you have several millions than maybe I would put it into Redis or something similar where I can utilize ready-to-use HyperLogLog cardinality estimation.
I was considering something like this. I guess I'm worried that saying I can't handle the workload will make me less likely to be looked at for new projects. Have you, or anyone else reading this thread, had success with a conversation like this and have any pointers?
Not me in person, but a colleague. The boss was very open minded and told him that he did not recognize that he was overloaded. It's important to differentiate between overchallenged and overloaded. You seem to be overloaded but not overchallenged - and this needs to be communicated to your prinipal (absolutely certain!).
You could use any kind of hardware token like RSA, Fortinet or many other manufactors. The usually offer a software version four your desktop as well. But usually you'll hate this additional piece of crap in your pocket. The combination of something in your mind (password) and something in your hand (smartphone) is the optimal setup. If you have a password or fingerprint to unlock your phone and the token generator app for 2FA it should be enough security.
If you're an engineer your need to stay up to date with the technical evolution: coding standards, frameworks, languages & trends.
If you switch into a management role, you need to be strictly organized (accuracy, punctuality, reliability), a good salesman for binding and motivating the project stakeholders AND all the engineering skills mentioned above.
I've switched into management 8 years ago and (even if it does not sound really nice) i need to know everything better than my envolved colleagues to decide if they're on track.
I'm currently taking a look at all the libraries and will test the functionality over the next days. Some of them are realy overweighted - especially the D3-based libs seem to be slower than the others.
