Storage: The DAS stack

Let’s talk about a typical Direct Attached Storage stack from a server system’s perspective.

Let’s cover the storage stack in both software and hardware layers, which can later help us bring in the networked storage concepts with ease.

Let’s take a quick look at the stack for ease of understanding.

The Storage Subsystem

The storage subsystem stack diagram has software components in blue  and hardware components in red. As with common sense, both are in just reverse order to each other. In a way, this stack might look just like networking stack. Let’s walk through the maze.

When an application requests for some operation a file, the request is passed on to the filesystem layers. Like in Linux, it’s possible to have a Virtual File System or VFS layer which will then switch to actual filesystem drivers to do the job. The filesystem then starts working through it’s magic on it’s internal data structures to figure out where and how to go about serving the request. One best example is looking up an inode and the indirect blocks. Let’s save some for detailed discussion later.

The filesystem then asks the disk accessing components, e.g. SATA or SCSI about the blocks it’s interested in. Once the request arrived in here, from now on we are only talking in terms of block numbers, be it LBA or some other mechanism. We don’t know anything about filesystem at all. Tell the block number and get it, read or write. The disk accessing subsystem then, looks up to do what best it can do, often re arranging the commands for a sequential operation improving the performance. Finally places the request packet to the HCI.

The host controller interface (HCI) or host bus adapter (HBA) in some cases such as SCSI drivers take in the packet , does the job of taking the series of commands for the disks, appropriate buffers and the disk ID as input, makes the queue up to place the commands to the disk along with disk ID. The appropriate queue is then passed on as a bunch of bytes/words to the BUS interface driver. In some cases, the HCI is rather really complex piece of software, such as for SATA, containing may layers such as transport layer and link layer within itself.

Finally the data to be sent on the bus arrives at the bus drivers. The bus drivers often are just simple ones to fiddle with a few flags and place the data on the bus, often with DMA or direct memory accessing subsystem and then let things go. When the DMA is done an IRQ comes up and informs the driver that the operation is complete. Here, the driver then returns control the HCI driver and HCI driver might wait for it’s own IRQ telling the command queue is sent successfully to the disk or just return the control to the Disk driver. Here the disk driver will have to wait for the disk job done and the IRQ tells that it’s complete. Only then the control can reach back to the filesystem telling that job was well done and the day is going good.

The skipped RAID component, let’s talk about it in a while.

Talking about the hardware, let’s start with the disks. The disks are just a few stores for the data and a few electronics glued to transfer data in and out. Often, disks take a single Logical Block Address and return the appropriate block by maintaining internal LBA to track/sector mapping. In the initial days, the disks just had a minimal electronics, but later the integrated drive electronics or IDE came in to picture. The same way, the Small Computer System Interface (SCSI) came in to existence. Though this is a dinosaur age story, that might just amuse you a bit. Though the SCSI is a common interface for the system, it’s mostly used for mass storage. Often, most storage systems emulate SCSI, such as USB pen drives. Then came along the AT Attachment or ATA (AT coming from PC-AT) and then came down the Serial ATA or SATA.

The SATA, however has it’s own protocol stack, though it supports a legacy mode in which the system can access a SATA subsystem as if it were PATA. As the SATA HCI can be operated in both backward compatibility mode with PATA and advanced HCI mode, the controller includes both standard IDE electronics & advanced HCI components.

Finally, the Bus interfaces. Most of today’s systems run on top of PCIe or Peripheral Component Interconnect – Express bus. In few words, it’s an advanced serial IO bus system that interconnects the CPU with the peripherals at a very high speed. Please visit http://en.wikipedia.org/wiki/PCI_Express for a quick look. Some posts on it a bit later.

In the next post, let’s talk about blocks, RAID and stripes.

Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: