Used to buy lots of things from Amazon. About couple of years ago, shifted to buying things from Costco and it's been a great experience. Everything I get is authentic, they only sell the best products and heard they treat their employees nicely. I stopped doing hours of review research to find out if the products are legit and if reviews were not fabricated.
I came to US about 10 years ago. Finished my PhD and spent about 4 years in Software Industry. Saved some money. US has taught me a lot about entrepreneurship and software. Now moving to India for good to work on my own startup, spend more time with my hobbies and to stay close with my parents who are in their 60's. It took me more than an year to convince my parents (hardest to convince was my Mom) and my Wife. Looking back, I feel that is one of the best, courageous and hardest decisions I have taken in my life.
Also thanks to YC for accepting my Startup in to their Startup School Advisor Track. Namaste!
It’s the right decision. You will be able to take what you learned home and create wealth for thousands, if not more. India is going to surpass China in the next century.
India is one of the hardest countries to do business in. On top of that, many businesspeople (not to mention the government) are dishonest and will happily run with your money. The "guanxi" situation outlined above very much applies in India.
I'd be curious to see how someone educated on American entrepreneurship performs.
India is going to be buckling under overpopulation, whereas China had already tackled that decades ago. Parts of India will be uninhabitable due to climate change, and there are already major water shortages.
However the Government does seem to be on the right track with their moves to introduce GST (replacing myriad taxes and duties from state to state) and cracking down on illegal cash.
We all know that diversity drives positive economic outcomes. India is by far the most diverse large country on the planet. That combined with her diaspora to the west and the consequent knowledge transfer, in addition to not being a major strategic rival to the USA, adds up to an extremely positive medium to long term outlook.
Yes there are major problems with corruption, it's true, but there are major problems with corruption in China and the USA for that matter. It's just in the USA the corruption is much more sophisticated and genteel. I mean this positively and I honestly believe Indian corruption is going to move toward the US model thanks to the large numbers of Indian nationals who have seen how much better that kind of corruption works. It's a classic case of not letting the perfect be the enemy of the less bad.
Do you have sources about diversity driving positive economic income ?
It seems to go without saying for you, but it's not that natural for me. There are several examples of very homogeneous countries, such as Japan or Korea, doing economical prowess.
The largest country in the world is Russia, and the richest is Iceland. Do you mean the US ?
If so, what makes you think the US got successful because of diversity instead of other factors ?
Similarly, there are studies showing that skill diversity is important, but not bio diversity. For example this meta-study[1] :
"Support was found for the positive impact of task-related diversity on team performance. In contrast,
bio-demographic diversity had no relationship with team performance"
Two things keep coming up while comparing GCP and AWS:
* This accomplishment would not have been possible for our three-person team of engineers with out Google Cloud (AWS is too low level, hard to work with and does not scale well).
* We’ve also managed to cut our infrastructure costs in half during this time period (Per minute billing, seamless autoscaling, performance, sustained usage discounts, ... )
This thread here came at the right time. Today whole day I attended the DynamoDB training. Honestly, one thing that I understood is its cost based on reads and writes per second. Irrespective of the amount of read data per operation (whether its 1 bytes or 100 bytes), its always charged for 1KB. So, as a work around what they suggested is using a Kinesis, a Lambda and an another service to make the write operation as a batch, in such a way the reads are near 1KB always. He pitched it like thats the perfect way to do. The problem I see is too many moving pieces for a simple thing to achieve. If the Dynamo team makes the reads cost based on the actual data we are all set.
* Kinesis Streams: Writes limited to 1K/sec and 1 MB / shard, reads limited to 2K/shard. Want a different read/write ratio? Nop, not possible. Proposed solution: use more shards. Does not scale automatically. There is another service called Kinesis Streams that does not offer read access to streaming data.
* EFS: Cold start problems. If you have small amount of data in EFS, reads and writes are throttled. Ran into into some serious issues due to write throttling.
* ECS: Two containers can not use same port on same node. Anti pattern to containers.
AWS services have lots of strings attached and minimums for usage and billing. Building such services (based on fixed quotas) is much easier than building services which are billed purely pay per use. This complexity + cost optimization pressures lead to complexity and require more human resources and time as well. AWS got good lead in Cloud space, but they need to improve their services without letting them rot.
Agree totally. The solution to overcome those shortcomings in AWS, is to sort of put bandaid's with more services (at least their suggestion). I do understand, its not feasible to provide service which fits for everyone, however it will be good if they solve the fundamental problem.
One more to add in the list.
In DynamoDB during peak (or rush hour) you can scale which increases the underlying replica's(or partitions) to keep the reads smooth. However, after the rush hour there is no way to drop those additional resources. May be someone can correct me, if I am wrong.
Thanks. Not the auto scaling part. I thought even manually if you scale up with new replica's, we can't scale down. I should read the manual and get a clear picture.
> * ECS: Two containers can not use same port on same node. Anti pattern to containers.
Could you elaborate for this? I'm not sure I understand, are you saying that 2 containers cannot be mapped to the same host port? Because that would seem normal, you can't bind to a port where there's already something listening. But I guess I must be missing something.
The OP is talking about how when using a classic load balancer in AWS, your containers will be deployed all exposing the same port, kind of like running "docker run -p 5000:5000" on each ec2 in your cluster. Once the port is in use, you can't deploy another of that container on the same ec2 node.
The solution is to use AWS's Application Load Balancers instead, which will allow you dynamically allocate ports for your containers and route traffic into them as ECS Services.
I'm not familiar with the details of AWS here, but maybe the OP means mapping two different host ports to the same port on two different containers? That's all I can imagine that would be a container antipattern in the way described.
That is perfectly possible with ECS, so I don't know what OP was referring to. The thing I remember though is that you have to jump through a lot of hoops like making 4 APIs calls (or worse with pagination) for what should have been a single call to make such a system work on ECS.
Nowaday you would often run containers with a container network(flannel, calico, etc.) that assigns an unique IP per container thus avoids conflicting port mappings regardless how many containers with the same port run on a single host.
Adding more context. Sorry for missing it out in first place. I mostly work in Big Data Space. Google Clouds Big Data stuff is built for Streaming / Storing / Processing / Querying / Machine Learning at Internet Scale data (PubSub / Bigtable / Dataflow / BigQuery / Cloud ML). AWS scales to terabyte level loads. But, beyond that, its hard and super costly. Google's services autoscale to Petabyte levels / millions of users smoothly (for example BigQuery / Load Balancers). On AWS, it requires pre warming / allocating capacity beforehand and that costs tons of money. In companies working at that scale, that usual saying is "to keep scaling, keep throwing cash at AWS". This is not a problem with Google.
Quoting from the article: "This accomplishment would not have been possible for our three-person team of engineers to achieve without the tools and abstractions provided by Google and App Engine."
Talking about the use case from the article, they release the puzzle at 10 and need to have infra ready to serve up all the requests. On AWS, you need to pre warm load balancers, increase the quota of your Dynamo DB, scale up instances so that they can withstand the wall of traffic, ... and then scale down after the traffic. All this takes time, people and money. Adding few other things author mentioned: Monitoring/Alerting, Local Development, Combined Access and App Logging ... will take focus from developing great apps to building out the infrastructure for apps.
Currently, I am working on projects that use both Amazon and Google clouds.
In my experience, AWS requires more planning and administration to handle the full workflow: uploading data; organisation in S3; partitioning data sets; compute loads (EMR-bound vs. Redshift-bound vs. Spark (SQL) bound); establishing and monitoring quotas; cost attribution to different internal profit centres; etc.
GCP is - in a few small ways - less fussy to deal with.
Also, GCP console - itself not very great - is much easier to use and operate than AWS console.
Could you please post the URL for the resource and the number of hits it receives? I'm interested in high load websites and I have a hard time picturing how this could lead to petabytes.
The impression I'm getting is not that GCP scales better, but it scales with less fuss - the anecdotes here all suggest that with AWS, once you hit any meaningful load (i.e. gigabytes), you need to start fiddling with stuff.
I don't know if this is actually true, I've never done any serious work in AWS.
Hold on, please do not say google cloud scales well, yes they do have services that make a ton of claims, but unlike AWS, things don't work as promised which is magnified by the fact that their support is way worse.
Additionally, Big Query is far more expensive than Athena, where you have to pay a huge premium on storage.
The biggest difference is that what amazon provides you in infrastructure, where as google provides you a platform. While app engine is certainly easier to use than elastic bean stalk, you have very little control over what is done in the background once you let google do its thing.
GCP support can sometimes be bad but these other claims don't add up. What isn't working as promised? BigQuery can do a lot more than Athena and it's storage pricing is the same or cheaper than S3.
We've used 5 different providers for a global system and GCP has won by both performance and price. We still use Azure and AWS for some missing functionality but the core services are much more solid and easy to deal with on GCP, which is also far more than just app engine.
It seems very strange to paint AWS with such a broad brush, considering that AWS has tons of services at various levels of abstraction (including high-level abstractions like Elastic Beanstalk and AWS Lambda).
Sorry to not add context. I was referring to the use case the author of article was talking about: running a website: You need to stitch: ELB / EC2 / Database / Caching / Service Splitting / Auth / Scaling / ... Where as on Google Cloud, App Engine covers most of the points.
Even better are Google managed services (PubSub / Dataflow / Datastore), which scale up and down based on usage (cloud native products) and thus save money automatically compared to their equivalents in AWS (Kinesis / Kinesis Analytics / DynamoDB) which does not autoscale.
It is also potentially due to Google owing their private fiber backbone that connects all regions and as well as their software Defined Network that allow high bandwidth and low latency routing of packets across regions.
I don't think so. GCP's bill is usually about 50% of AWS's bill for same application, if you run it full hour (from my personal experiences and from several others as well: https://thehftguy.com/2016/11/18/google-cloud-is-50-cheaper-...). GCP has lot more cost saving features like seamless scalability, custom shapes, sustained discounts and so on. If you workloads span less than hour, GCP can offer more then 50% savings.
I refuted some of the networking claims in that article previously (I work for AWS). Especially the bizarre claims that you have to get a C4.4xlarge for 1gpbs... The 220 mpbs network cap claim is just not true. Just run iperf3 on any aws instance to a GCE instance and you can see greater than 220mpbs.
Honestly we all know that the small instances have terrible CPU that doesn't let you use the advertized 1Gbps anyway. Other than that, even if AWS let 1Gbps traffic go on for a while, you get throttled pretty quickly from my experience.
> Just run iperf3 on any aws instance to a GCE instance and you can see greater than 220mpbs.
For how long is the question. Historically, it’s been considered common knowledge (might just be an urban legend) that AWS, even if you pay for more traffic, at some point just throttles you, the same way that they do with IO.
AI is the future of Computing. TensorFlow has established itself as the standard for machine learning. It has an amazing community, and love from researchers as well. TensorFlow, along with Kubernetes, are positioning Google Cloud as a strong contender in Cloud Computing Space. With Cloud TPU and Cloud ML, Google has leaped few years ahead of other AWS and Azure.
But the funny thing is that lock-in is very low in this space, at least for the actual ML part. 90-99% of the work in ML is in acquiring and cleaning the data, feature engineering and so on. Once you have your training data in good shape, the ML engine is basically pluggable - the only barrier to switching from Azure, Google, AWS etc is the cost of moving the data, if you store it "in the cloud" in the first place. The winner won't be the slickest API - it will be whoever makes it easiest to migrate your data in, whether that's a traditional DW, sensor data feeds or whatever. If you keep the data on-prem you could switch with relatively minor effort.
I'm not sure how much work you do in this area, but the only ML models that are pluggable and easily moved are ones that are relatively outdated, widespread, and thus provide little benefit over your competitors.
ML moves fast, and to keep up, you need to be using a flexible yet powerful framework or you will run into major slowdowns in implementation time. Tensorflow is that framework, and, by the way, it also happens to have the best support for actually deploying your models across datacenters.
Migrating your data into some cloud provider is irrelevant and consumes maybe 3% of the total effort involved. Data is maybe 10%, and the rest is developer/research time.
I disagree with this. In my experience, improving your dataset quality (either via gathering more examples, getting more reliable labels, or doing better normalization) has yielded bigger gains than trying to use the latest and greatest model architecture for the problem. Linear and tree models are still widely used in applied problems.
The quality of the ML model - provided is of the right family in the first place of course - might move you a few percent. The quality of the data you feed it - by an order of magnitude or more. That's why real data scientists spend the vast majority of their time on prep work.
The comment is a fun poke. Please don't take it literally! NVidia's new Volta GPU has Tensor cores that I think are very similar to what was in first generation TPU. I read the TPU papers and blogs published recently and I am seriously impressed by TPUs!
