For cost on AWS, use S3. AWS price local storage on VMs to make HDFS not competitive. If you want the HDFS API but S3 storage costs on AWS, use HopsFS-S3 - https://www.logicalclocks.com/blog/hopsfs-100x-times-faster-... (disclosure: work on HopsFS).

HDFS provides POSIX-like API, and now has atomic metadata operations that S3 doesn't (mv/rename, chown, chmod) and append for files.