Most of my experience mirrors that found in other comments, so here's a few unique quirks I've personally had to work around:
- You can't trigger Lambda off SQS. The best you can do is set up a scheduled lambda and check the queue when kicked off.
- Only one Lambda invocation can occur per Kinesis shard. This makes efficiency and performance of that lambda function very important.
- The triggering of Lambda off Kinesis can sometimes lag behind the actual kinesis pipeline. This is just something that happens, and the best you can do is contact Amazon.
- Python - if you use a package that is namespaced, you'll need to do some magic with the 'site' module to get that package imported.
- Short execution timeouts means you have to go to some ridiculous ends to process long running tasks. Step functions are a hack, not a feature IMO.
- It's already been said, but the API Gateway is shit. Worth repeating.
Long story short, my own personal preference is to simply set up a number of processes running in a group of containers (ECS tasks/services, as one example). You get more control and visibility, at the cost of managing your own VMs and the setup complexity associated with that.
IIRC, it's a philosophical argument that prevents it - SQS is pull, Lambda triggers off push. They don't want to add a push mechanism to SQS, ergo they can't support Lambda.
- You can't trigger Lambda off SQS. The best you can do is set up a scheduled lambda and check the queue when kicked off.
- Only one Lambda invocation can occur per Kinesis shard. This makes efficiency and performance of that lambda function very important.
- The triggering of Lambda off Kinesis can sometimes lag behind the actual kinesis pipeline. This is just something that happens, and the best you can do is contact Amazon.
- Python - if you use a package that is namespaced, you'll need to do some magic with the 'site' module to get that package imported.
- Short execution timeouts means you have to go to some ridiculous ends to process long running tasks. Step functions are a hack, not a feature IMO.
- It's already been said, but the API Gateway is shit. Worth repeating.
Long story short, my own personal preference is to simply set up a number of processes running in a group of containers (ECS tasks/services, as one example). You get more control and visibility, at the cost of managing your own VMs and the setup complexity associated with that.