The Exadata uses Flash, lots of Flash and it uses Infiniband, not Fiber Channel, Ethernet or FCoE. Exadata uses PCI Flash memory modules not Solid State Disks. Actually it is not a general purpose storage controller and it doesn’t support non Oracle Database workload. It is designed from the outset to host Oracle Databases and deliver 1M IOPS.

There seems to be a mutilevel cache, RAM in the Sun Galaxy Intel 5500 servers, Flash in their PCI bus and then SAS storage attached via controller. Oracle use a new advanced compression technology to deliver ultra fast transfer rates between the Database Servers and the Infiniband attached storage contoller, and all the way down the line via the Flash and SAS drives.

Larry Ellison thinks that Flash is a major disruptive force in the storage industry and wants to take the lead in this technology. Therefore he decided that we will only sell Exadata V2 with flash even though it raises the price.

Oracle claim to have achieved 50 GB/sec of bandwidth from the Exadata flash as it is attached to the PCI bus rather than to the disk controller. I wonder if the flash is a Fusion-io product. It’s not announced just a guess.

We have implemented a smart flash cache directly in Exadata storage. The Exadata Smart Flash Cache allows frequently accessed data to be kept in very fast flash storage, while most of the data is kept in very cost effective disk storage. This happens automatically without the user having to take any action. It is the ultimate ILM (Information Lifecyle Management) story. Our flash cache is smart because it knows when to avoid trying to cache data that will not fit in the cache. Also, we allow the user to provide directives at the database table/index/segment level to ensure that specific application data is kept in flash. Tables can be moved in and out of flash with a simple command, without the need to move the table to different tablespaces/files/LUNs like you would have to do with traditional storage with flash.

Oracle seem to be using a tightly integrated database compression system they are calling Exadata Hybrid Columnar Compression that allows more data to be stored on fewer disks and the data to be moved around faster.

Interesting!