For Great #StorageGlory : Wargaming 4K Capture

ride

This week at work, I’ve been spending a bit of time outside of my cramped IT office (home to the #InfratructureGloryWhiteboard & the #DrawerofTears where hardware bits ‘n bobs and dreams go to die) in another part of the building: the movie theater.

Yeah, we got one at my work. And it’s as cool as it sounds.

I’ve been hanging out down there to lend a hand to some of our in-house technical experts on a different type of storage problem. That’s understating it actually. It’s a radically different storage problem.

The conversation went like this:

“Hey Jeff, we have some disk. It’s in a rack-mounted box and we want to do X with it, and maybe Y. Want to have a look-see?”

“Do I?!? You had me at disk. I’ll be down quicker than you can say Microsoft Lync 2013.” 

I think I actually skip-hopped my way down to theater.

I can’t go into too much detail here but let’s just suppose:

  • You or your business own or rent professional-quality, high-end motion-picture cameras & lenses that are capable of capturing the world and all that is in it in stunning 4,096 x 2160p format (commonly referred to as 4k)  at certain frame rates
  • Let’s further suppose you (a different you now) are a creative type and like such cameras and want to point them at famous thespians in remote & hard to reach locations
  • Now let’s suppose finally that said thespians are expensive & their time is precious, and that you will have limited opportunities to re-shoot scenes starring said thespians

Understand that I’m not being asked to build such a system; it’s more like I’m being asked to help some sharp industry guys think about storage performance in that context.

It’s fun thought experiment compared to what I’m used to. Some lunch napkin doodle compare/contrast:

[table]

Datacenter Storage, On-Set Storage

Should be flexible, Can be purpose-built

Speed second fiddle to data integrity, Speed & data integrity paramount

Can leverage existing infrastructure with iSCSI SAS or FC, On set there is no infrastructure apart from what you bring

Needs to be flexible for all sorts of IO, Should do one thing and do it well- sequential IO

Is loud & full of fans and rotational disks/SSDs, Should have lots of capacity but be as silent as possible

Needs to be built well and to spec, Needs to be hardened and tolerant of rough environmental conditions

Rack upon rack or density-focused IOPS/U, Must be easily transportable

[/table]

Oh, and it needs to be fast. Really fast. Look at the performance bogies I’m working up:

Based off the excellent video bitrate calculator by Peter Forret
Based off the excellent video bitrate calculator by Peter Forret

Right.

So I just have to think of ways to write sequential uncompressed data to an array at a rate of 3,200 megabytes per second within the constraints listed above.

And it goes without saying the mechanics of the storage system as well as the file system itself need to guarantee that the mad 4k inbound IO actually gets committed to rotational disks because this isn’t your run-of-the-mill financial data, or VDI user profile, or SQL table, this is art people..

Hell yeah. By Monday then? Got it!

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s