Hi there ehutch79, just a quick clarification. Cloud Files is object storage, whereas Cloud Block Storage is block storage. The performance characteristics of both types of storage are very different. One way I use sometimes to explain the difference is by talking about how they are accessed: Cloud Files objects are accessed using HTTP (think REST), whereas Cloud Block Storage blocks are accessed using low level OS I/O operations (think block read and write operations). Because of that, you could implement a database on Cloud Block Storage, but not on Cloud Files as it would not be very performant. Think Cloud Files when you have a website that needs media, large objects, application-specific content, files, etc., or when you need CDN for improved performance at the edge (CDN is a great feature of Cloud Block Storage). Think Cloud Block Storage when you could have used a regular old hard drive: you provision it, you format it, you may stamp a file system on it, or use the block storage for MongoDB or a relational database (with the difference that with Cloud Block Storage you can pick Standard drives or Solid State Drives).
Hey jrarrendondo, I've been using Rackspace for a few months now, using some of your servers and Cloud Files and I'm loving all of it. So what you're telling me is if I sign up for Cloud Block Storage, I could potentially "attach" it to my server as another hard drive? I know you guys are really great with your documentation and announcements of potential ways of using the storage, so if you could point me in the right direction, I'd be interested in getting this set up.
Inability to add more disk was a huge problem when were at slicehost [now rackspace]. Glad to see that finally lifted.
I was under the impression that it wasn't wise to run a high read/write database on SSD because they have limited writes. But obviously people are finding it wise to do this. Is a 20GB postgresql going to work ?
I guess considering the number of times I've had to migrate servers it shouldn't be a problem lasting 5 years ;)
How many customers asked you for a network block device? I can't think of anyone who uses raw block devices these days. Qmail? Some db storage engines?
I'd posit that what most consumers actually want is a persistent filesystem. Clear consistency semantics is probably a bonus at this point.
Vending a fs shim to your domus eliminates a whole mess of abstractions.
You realize this is essentially the same offering as EBS? It just shows up as a raw device in your operating system and you can create your preferred FS on it and its a persistant file system. Some people might want to use xfs some might want to use zfs so just providing a raw block device is the best way to have widespread compatibility.
I ask "How many customers asked you for a network block device" but I don't know what EBS or a "raw device" is?
So yes, I'm well aware that "compatibility" is the general reasoning for building a block device interface. But how many unique domu kernels do these providers support? Two, maybe three on the outside? And minus win32 theyre all posix with shockingly similar vfs interfaces.
Which comes back to Henry ford and his faster horse. Do customers actually want another layer on the abstraction fest so they can stack their mount option of choice on top? Or do they want a persistent file system with well defined consistency semantics.
I've never heard a customer actually request Yet Another Leaky Block Device Abstraction. And if that customers out there, what's their use case? Because building an fs shim to the dom0 seems to eliminate a whole mess of underlying infrastructure. So why isnt anyone doing that?
Remember Koenig’s Fundamental Theorem of Software Engineering: “We can solve any problem by introducing an extra level of indirection”? :-)
You would be surprised if I could tell you the number of customers that asked about Cloud Block Storage. This is one of the most requested services! Since all storage is block storage at the lowest level of abstraction on top of the device, it is always used by higher level abstractions. The question is whether that level of abstraction is appropriate for your application and your productivity as a developer if writing new apps. I see block level storage as the most versatile because it is the most fundamental and you can use it for anything: storage for emails, storage for tables and indexes of your favorite database, storage for file systems, archives, etc., by building that abstraction on top of blocks.
But I agree with the basic idea of your comment: as the most fundamental layer, then it may also be too low an abstraction layer to be used directly if you are writing a new high level app (think a business app for instance). That’s why there are file systems, databases, etc., all of which use blocks to do their thing and provide higher level functionality. I like to use the example of how most people don’t stop to think that systems really do not know about files and directories, or database tables and indexes. They just know blocks. But the file system abstracts blocks and provides the appearance of files and directories and hierarchy. And DBAs and developers using our Cloud Databases product are happy to do SELECT A, B FROM T WHERE C=’value’ without having to worry about blocks. But blocks are always there, underneath all that.
Whether you would want to stay at that lower level of abstraction depends on what you are doing. The higher the level the more specialized the abstraction, the more flexibility you gain at that level and the less flexibility you have to optimize at the lower level. For example, on average, 50% of the last block used by a file is wasted in many file systems. Nothing you can do about that. But I bet those directories and ability to find a file by name come in handy at that higher level.
I think that there are two cases where block is needed. The first one is to deploy commercial applications, databases, file systems, etc. for which you have no choice because you did not write the app. The second is for optimization of storage for infrastructure applications you are writing and have control of and where you want that optimization.
Many people writing business applications or web apps will be happy to use object storage or Cloud Databases for MySQL. No need to worry about block level because we abstract it for them. Many people will need a file system: they will just create a block storage device and format it as an fs and move on with their lives. Others will use it as a database. Others will write apps we cannot even imagine right now. But blocks will be there, humbly, doing their work.
Hi there ehutch79, just a quick clarification. Cloud Files is object storage, whereas Cloud Block Storage is block storage. The performance characteristics of both types of storage are very different. One way I use sometimes to explain the difference is by talking about how they are accessed: Cloud Files objects are accessed using HTTP (think REST), whereas Cloud Block Storage blocks are accessed using low level OS I/O operations (think block read and write operations). Because of that, you could implement a database on Cloud Block Storage, but not on Cloud Files as it would not be very performant. Think Cloud Files when you have a website that needs media, large objects, application-specific content, files, etc., or when you need CDN for improved performance at the edge (CDN is a great feature of Cloud Block Storage). Think Cloud Block Storage when you could have used a regular old hard drive: you provision it, you format it, you may stamp a file system on it, or use the block storage for MongoDB or a relational database (with the difference that with Cloud Block Storage you can pick Standard drives or Solid State Drives).