Sandwich News

Office River Timelapse

Thu, 04 Dec 2025 18:24:34 -0700

I was wondering why one of my AWS accounts had almost 1TB of data in S3. Turns out I never cleaned up the images from Shipping News -- a project where I captured a photo every 30 seconds from my office window, saved it to S3, and ran image recognition to detect boats passing by on the Hudson.

I've been doing a lot with ffmpeg lately, so I thought I'd see if I could make a quick timelapse. This is just the first day I captured, but I think I have about a year of data...

Hudson River Window Aug 13, 2022

It's been offline for a couple years now. I started getting lots of false positives, so I "temporarily" turned it off.

I've still been meaning to take the images with detected boats and try some labeling/training with YOLO. I'd love to bring it back online if I can get more reliable detection.

It would be cool to do a full year timelapse with detection -- slowing down when boats are crossing the screen, and speeding up when not much is happening.

This is what makes me a digital hoarder. Now I'm never going to be able to delete these images!

Yosemite Photogravure

Fri, 21 Oct 2022 10:00:00 -0700

Me showing off a print (photo credit Irene Searles)

I had an incredible time earlier this month learning how to make Photogravure prints at the Ansel Adams Gallery in Yosemite.

We had a fantastic instructor, Clay Harmon, who wrote the book on Polymer Photogravure.

It really was an amazing experience waking up every morning in Yosemite, walking to class (occassionally seeing bears on the way!) and spending the day making prints in Ansel Adams' darkroom - I felt like a real artist!

We even got the opportunity to spend some time exploring a few locations in the park to take photos, guided by Michael from the gallery who was an amazing guide and teacher.

I feel like I'm hooked on the process.

Below are some of my prints from the workshop.

Fire Island - Photogravure print 2022

Golden Gardens Park - Photogravure print 2022

Tappan Zee Bridge - Chine collé photogravure print 2022

Ballard Locks - Photogravure print 2022

Tappan Zee Bridge - photogravure print 2022

Sedro Wolley Barn - photogravure print 2022

I got to make a ton of prints. None of them are perfect, but I am ready for more practice!

It's really such a fun process that I can't wait to try again.

PyCon 2022

Fri, 03 Jun 2022 18:00:00 -0700

Videos from PyCon 2022 are online. I've only made it through a few so far, but they've been great. I thought I'd write up a quick summary and some thoughts on the sessions I've been able to catch.

Brandt Bucher: A Perfect Match The history, design, implementation, and future of Python's structural pattern matching

Talk Video

Brandt is the author of the specification for adding Pattern Matching to Python (in 3.10). The original spec PEP 622 was later replaced with a series of PEPs, one for the spec PEP 634, one for the motivation and rationale PEP 635, and an introductory tutorial on using pattern matching in python PEP 636.

I'm a big fan of pattern matching from my scala days and I'm excited to see it land in python. The implementation was obviously inspired by other languages, but in a way that feels at home in python.

Pattern matching makes it really easy and clear to pull apart and match against data structures.

An example from the tutorial:


@dataclass
class Click:
    position: tuple
    button: Button

match event.get():
    case Click((x, y), button=Button.LEFT):  # This is a left click
        handle_click_at(x, y)
    case Click():
        pass  # ignore other clicks

You can see how easy it is to destructure the Click dataclass and use its attributes in our match.

I'm really excited this has landed in python and can't wait to start using it in my own code.

Fred Phillips: Hooking into the import system

Talk Video

I've never really thought about it, but the import system in python is dynamic and you can provide your own import resolvers fairly easily. I guess it's somewhat similar to something like Classloaders in Java.

In this talk, Fred walks through simplified versions of real world import hooks he's written/used in production, including a "Blocklist" loader (raises errors if the import is included in a blocklist), and a DB Loader (loads python code from the DB on import), as well as a tracing loader that prints out imports as they are happening.

Looking into this more, I discovered an example import finder implementation on Real Python that attempts to pip install modules as they are being imported.

While loading code from a DB kind of scares me, it was a really interesting talk and this is something I definitely want to play with more.

Related there's a draft PEP for making imports lazy PEP 690.

Jeremiah Paige: Intro to Introspection

Talk Video

Python has always been interesting to me w.r.t. how much information is available to inspect at runtime from your code.

For example, given a method with a docstring comment, like:

def my_method():
    """
    This method does something amazing!
    """
    pass

One can access the docstring with my_method.__doc__ and see that comment. You can even access the source by using inspect.getsource(my_method)! This kind of stuff blows my mind coming from other languages. Obviously there's a lot more useful stuff you can get including typing information, call signatures, etc. In this talk, Jeremiah walks through the basics of introspection in python, with an eye towards demystifying how the python help command could be implemented. Definitely worth checking out!

Anthony Shaw: Write faster Python! Common performance anti patterns

Talk Video

Anthony also discusses perflint a linter that he's written to check for some of these performance anti-patterns. As he's mentioned in the talk and the readme, it's currently pretty early and will raise false positives (can confirm), but it's a great idea and definitely worth checking out for any performance sensitive code.

Bruce Eckel: Making Data Classes Work for You

Talk Video

Bruce talks about encapsulating validation logic into your Data Classes using __post_init__() (and frozen=True).

As an example, he goes through using a Stars dataclass, that basically wraps an int, but restricts it to be in some range. Then any functions that take or return Stars are easy to reason about.

It seems like a good strategy for ensuring consistency and safety in your code.

Classes are not types, but it reminded me a bit about phantom types in Scala.

More Cyanotype Toning

Mon, 07 Feb 2022 10:00:00 -0700

I've been slowly working on a series of photographs of the Tappan Zee Bridge that I've wanted to print as cyanotypes.

The images really lend themselves to monochrome, and there's something about the blue of the water and the cyanotype blue that seems to work. That said, I did want to play around with some toning to introduce additional visual interest.

I've experimented with cyanotype toning before, but I thought this would be a good opportunity to try out some of the toning techniques from Annette Golaz's excellent book Cyanotype Toning.

This is really an amazing book. If you're working on cyanotypes at all, you have to pick up this book. Besides the great technical content, the work shown by Annette and other highlighted artists is incredibly inspirational.

I've been really happy with most of these. There's still some details I want to improve, but overall I like the direction they are going.

tappan zee bridge, cyanotype, pink pepper

This is one of my favorites. Pink Peppercorn (which is actually a berry!), produces a grayish tone. I think I got really lucky with the split tone that occurred here - the gray in the clouds while leaving some blue in the sky and water is just great.

Below is an untoned version for comparison.

tappan zee bridge, cyanotype

tappan zee bridge, cyanotype, oak galls

Oak Galls produce a nice dark black color. I think this ended up a little too dark, but I really like the inky black water.

Below again is an untoned version for comparison.

tappan zee bridge, cyanotype

tappan zee bridge, cyanotype, oak gall

Here's another w/ Oak Galls, again a bit dark.

tappan zee bridge, cyanotype, logwood

Logwood I thought would be darker, maybe it needs more time. To me it looks a little washed out, but, maybe it's worth more exploration.

tappan zee bridge, cyanotype

Here's a bonus bridge cyanotype untoned.

360 camera

Sat, 18 Sep 2021 17:00:00 -0700

I've started playing around with a 360 camera, the instax 360 one x2.

Tappan Zee Bridge, 2021

It has two super wide lenses on either side of it, that get stitched together to produce a 360 image.

Tappan Zee Bridge, 2021

The coverage of the lenses actually overlaps enough to hide the camera (and selfie stick from view).

To produce an interactive 360 view like above, the photos are stiched together into a single image using an Equirectangular projection (which you've probably seen used for maps of the earth).

Tappan Zee Bridge, 2021

The projections themselves are sometimes interesting as photos on their own. I want to continue to experiment with this. It's interesting to me that they appear so heavily manipulated, but in some sense are "straight out of the camera" (not exactly, but still..)

Tappan Zee Bridge, 2021

It feels like it could border on gimmicky if overdone, but I'm hoping to include a few of these in an upcoming photo zine I'm working on.

110 film

Sun, 10 Jan 2021 10:00:00 -0700

dobbs ferry train station, Feb 28th 2020

I picked up a 110 film camera after seeing that lomography produces film for 110.

Fujica Pocket 200F

The Fujica Pocket 200F was produced in 1977 and features a 29mm fixed focus lens at f11, and a mechanical shutter speed of 1/60. There are no exposure controls.

negative size comparison

These negatives are tiny, about half the size of 35mm.

Overall, it was a fun camera to shoot! Since there are no exposure controls I just pointed and took the photo, hoping for the best, with mixed results :).

Some of the shots came out nice, and it was fun to get the contrasty, super grainy look. It made me want to go back to shooting more film - and that's aways a good thing!

Check out some other 110 cameras at the Sub Club.

somewhere in nyc

half moon, dobbs ferry

Satsop Nuclear Power Plant

Tue, 21 Aug 2018 10:00:00 -0700

satsop nuclear plant, 2018

The Satsop Nuclear Power Plant was part of a larger effort to provide power to the Pacific Northwest. Construction started in the late 1970s and was ultimately abandoned before completion.

These days it's a business park, with normal office space, big warehouses, as well as some interesting use of the tunnels and buildings for fire/military training, movies, etc.

I went as part of a photo outing with pcnw a few months ago. It sounds like it's pretty easy to schedule a group tour of the buildings (with some cost involved) if you are in the area and want to check it out (it's a few hours south of Seattle).

Since we were moving to New York, I thought this would be my last big photo trip in Seattle, so I brought way too many cameras, including my huge 4x5, which I regret lugging around (although I still haven't developed the film, so who knows!).

I was pretty happy with how my instant photos came out! I still have a ton of fun with the instax, and it's so nice to have the prints right away to share.

satsop nuclear plant, 2018

Instant Photography

Tue, 10 Jul 2018 13:45:00 -0700

near centennial park, seattle, 2018

I recently bought a cheap Fuji instax camera for an upcoming road trip. I had been wanting to try out the instax film and I thought my 6 year old daughter would have fun documenting the trip with me.

The ergonomics of this specific camera are terrible, but having the prints instantly make up for it.

Most of the fuji instant cameras are pretty basic with not a lot of control and cheap lenses, but it's exciting to start to see some 3rd party cameras coming out, like the Mint InstantKon.

Overall, it's a lot of fun to use and cheap enough to just go for it.

My daughter shot most of the color film right away, but wanted nothing to do with the black and white.

I took the camera out on a photowalk with a friend to Centennial Park here in Seattle and I was pretty happy with the results. The black and white is a little high contrast, but it has a good look.

near centennial park, seattle, 2018

Toning Cyanotypes

Tue, 12 Jun 2018 20:45:00 -0700

cyanotype and tea, golden gardens, seattle, 2018

For PCNW's Longshot this year, I worked on a series of cyanotype variations.

I really love cyanotype printing. It's one of the safer photographic processes, you can coat the paper in light, you expose in the sun, and otherwise you just need water to rinse.

Sometimes, though, you get tired of blue!

MP Photography has a great overview and examples of toning cyanotypes. It's pretty easy (if time consuming).

The weather was a little weird when I made these: switching between overcast, rainy, sunny, cloudy, so I had a difficult time with exposure. Luckily, toning also seems to be a good way to rescue prints with exposure issues.

As part of the process, you bleach the print, so, it's ok if the print is overexposed a bit. There's also typically some staining, so it can help a bit with midtones and highlights if underexposed.

The above tea print is my favorite of the bunch. It was a bit underexposed, so I didn't bleach it, but just soaked it in tea for 3 hours. I think the tea tone works well for the photo, and I like the blue border that remains.

cyanotype and coffee, golden gardens, seattle, 2018

I do really like the cooler, closer to black look you get from coffee toning. This was a bit overexposed. I bleached for a bit, then 3 hours in coffee. I'm tempted to try for longer next time.

cyanotype, golden gardens, seattle, 2018

For comparison, above is a normal cyanotype print with no toning.

Part of the fun of these analog processes is the one of a kind results and ocassional surprises. It's also been a fun reminder to not worry as much about mistakes or imperfections and just embrace them.

The original image I used to make these prints was shot on color film with an old folding camera that had light leaks (see below)!

I can see the artifacts of the leaks in the final prints, but it's a lot less noticable and doesn't bother me at all.

golden gardens, seattle, 2018

Large Format Junkyard

Thu, 31 May 2018 16:50:00 -0700

paper and acrylic detail, junkyard, 2018

I had become interested in large format photography after a workshop with Daniel Carrillo on Wet Plate.

My friend, Hank, had given me a large format monorail camera, and I had taken it out a few times. It was exciting and fun, but also frustrating and my results had been mixed.

When I saw Daniel Gregory was teaching a Large Format Film and Lighting class at PCNW last winter, I was on board.

Here are some 11x14 darkroom prints from a recent junkyard trip (a mix of Ilford Matte and Ilford Warmtone fiber papers).

I've really been enjoying the process. While there are still frustrations, after repeated practice things are a lot smoother. I really love the simplicity and thoughtful intention that seems to come with shooting large format.

stack of bumpers, junkyard, 2018

dashboard, junkyard, 2018

portrait of a door, junkyard, 2018

Raspberry Anthotypes

Fri, 07 Jul 2017 11:20:00 -0700

Raspberry anthotype print, 3 days exposure

I've been paging through the excellent Book of Alternative Photographic Processes by Christopher James.

Anthotypes caught my eye as something fun and easy to try. It's an early process that uses flower or plant material (the name is from the greek anthos meaning flower).

If you've ever noticed paint or fabric washed out from spending a long time in the sun, that's basically the process!

There's a good step-by-step set of directions with photos on the Alternative Photography website, but basically:

mash up some flowers, plants, or fruits to produce a liquid.
- optionally add alcohol to help with extraction, or vinegar (supposedly darkens?).
- optionally strain out seeds, etc.
coat some watercolor paper with the liquid and let it dry
place something on the paper to block the sun where you want the image to appear
- the color of the paper will be the darkest area of the print
- anything uncovered will be lightened by the sun
  - unlike a 'typical' process, you want a positive image, not a negative
- you can use cutouts, plants, etc. or print a black and white image on a transparency
- a clip photo frame will help keep everything in place
put it in the sun, and wait!

Raspberry anthotype prints, in progress

Exposure times seem to vary greatly depending on the plant material used.

For my prints, I used raspberry with some alcohol and put them out in the sun for 3 full days. They probably could have used another day or two, but I got impatient.

Raspberry anthotype print

As far as I know, there's no known way to fix the images (make them permanent). Over time and with more UV exposure the images will eventually fade.

Anthotypes are a very old process (1842), but they never became that popular because of the long exposure times and image impermanence - there's still a lot to explore!

It's a really fun, easy, safe and environmentally friendly photographic process. I'd definitely recommend giving it a try!

I'm looking forward to trying it more with other plants. Blackberries and spinach look like they produce good results with reasonable exposure times. Onion skin looks really fantastic, but seems to take a month or more (I'm not sure I have the patience required).

Lumen prints

Tue, 13 Jun 2017 11:38:00 -0700

Lumen print, fennel

As part of this year's PCNW Longshot event this past weekend, Gina White and others were leading analog printing activities with Cyanotypes and Lumen prints.

Jill Enfield has a good overview of the process, including some really interesting examples.

Basically, it's just putting photo printing paper out in the sun. These prints should not be developed (they will just turn black), but can be fixed. The fixing will wash out a lot of the color, but make the print stable. Gold toning seems to work well.

Lumen print, garden flower

Different paper will produce different results, and you can use old/expired or even fogged paper or film.

The prints here I made were all done on Ilford Warmtone paper. Note that it's just ordinary black and white photo paper, but it gets really vibrant colors when left in the sun. These prints were gold toned, and no fixer was used.

Lumen prints in progress

Online I've found people recommend 20-60 minutes for fiber paper. Gina recommended much longer times, 5-6 hours. The color did seem to become more vibrant as time went on.

I did get impatient and my prints were only exposed for 3 hours. I'd like to experiment more on time.

Lumen print in progress

Moisture (from the plant) seems to play a big part in the color shifts.

Some of the detail from these I find pretty interesting, there's almost a 3D feeling. I'm not sure how much is a result of the toning vs. the print.

flower detail

fennel detail

I'm not that crazy about the prints I made, but it was fun and easy. I'd definitely like to try them again and see if I can get better results.

Give it a shot if you haven't already! All you need is some old photo paper!

Hand Coated Photography

Tue, 09 Aug 2016 21:09:00 -0700

Sedro Wolley Barn, Van Dyke Print

This past weekend I was fortunate enough to take a workshop on some historical photo processes at pcnw.

You can check out a few of my scans on flickr.

We covered both Cyanotypes and Van Dyke.

For both of these processes you apply the chemicals necessary for photosensitivity by hand on paper.

I already like making photo prints by hand, and I think this lends even more to that process. As a 'modern' photographer, I hate to admit that I like even the visible brush marks, but I really do!

Cyanotypes

Sedro Wolley Barn, Cyanotype Print

The cyanotype process is especially interesting to me. This was originally used for reproducing drawings, and is how blueprints got their names (they were cyanotypes).

There are also some incredible early images by Anna Atkins published in 1843 using cyanotypes for images of seaweed and algae.

Yes, the chemicals are toxic, but seemingly less-so than almost any other photo process?

The 'developer' is just plain water...

You don't even really need a darkroom for this process, just no UV light. And to develop, you can use the sun (though it may be unpredictable).

Exposures take a long time (20 minutes), but, that's par for the course here.

Once you coat the images, it's pretty smooth sailing.

Van Dyke

I honestly like the look of this process more than cyanotypes... But, it feels a little heavier. It requires a real fix, and thus an archival wash... It's more like making a silver print.

Future work

Even though I think my cyanotypes weren't especially successful, I feel like it is a process I really enjoyed and would like to explore more. The strong blues don't work everywhere (although you can modify the tone a bit with other bleaching/toning processes.)

Look for more here or on my photo site on that subject :).

Pushing bytes is expensive

Wed, 16 Mar 2016 16:05:00 -0700

In this post I'll be comparing cloud hosting costs, which I'm sure will be hopelessly out of date in just a few months.

I'm writing this post to try to solicit feedback on cheaper file transfer options. If you have any advice or feedback, I'd love to hear about it!

Maybe it should have been obvious, but I've been somewhat surprised at the high cost of transfer vs. storage when you start talking 10s of terabytes.

For example, storing 10TB of data in S3: $302.60, transferring 10TB of data out of S3: $921.51.

I've also been (pleasantly) surprised at how aggressive the AWS/CloudFront (CDN) pricing is -- at $0.085/GB (US only), it appears to be the cheapest way to get data out of AWS. It's cheaper than both S3 directly ($0.09) and even cheaper than serving directly from an EC2 instance (also $0.09). This is kind of amazing considering all the benefits of the CF service vs. doing it yourself, or even S3.

I've been working out the details for a SaaS app that doesn't use a lot of storage, but has the potential to have a lot of subscribers. So, it will use say 200MB of storage, but need to serve that data potentially 50,000 times each month. The S3 costs are neglible, a few cents, but even the cheapeast CF tier (US only), we are talking $870.40/month.

I love the CloudFront feature set and service and will continue to use it where it makes sense.

For this app, I will have a hard time being competitive with other apps in this space with those costs, so I've been looking for cheaper solutions. Below is a list of what I've compiled. I'm not considering any "free tiers" for the comparison. These are great for development/etc., but I think not relevant in an ongoing cost comparison.

Provider	Transfer Cost/GB
AWS/CloudFront (US)	$0.085
AWS/S3 (us-east)	$0.09
AWS/EC2	$0.09
Google Cloud Storage	$0.12
BackBlaze/B2	$0.05

Other Options

CloudFlare

CloudFlare is a pretty full featured CDN that actually even offers a "free" tier.

I signed up for a "pro" account to test it out. The problem is that even for the pro and business levels, there's absolutely no record provided of file access. That requires an Enterprise/Custom plan, where the pricing is determined by the sales team. I haven't yet made a call to a sales rep to determine what final pricing would be for my use case.

The other tricky thing here is that I still need to separately maintain and pay for an origin, the costs of which are entirely determined by how effectively CloudFlare can cache my content. What I mean is, if I put CloudFlare in front of CloudFront, and I'm only getting 50% hit rate from CloudFlare, I'm still paying for $435/month from CloudFront on top of what I'm paying from CloudFlare (that's a lot of CFs).

Digital Ocean

From what I can tell, they don't offer a straight cloud storage solution, but their "Most Popular" plan features 3TB of data transfer included for $20/month.

This works out to $0.00666/GB. Of course, I would like some basic redudency, so, say two machines and a load balancer, works out to about $0.01/GB/month? (($20*3)/6000), which is by far the cheapest I've seen.

Of course, there's quite a bit more there to build and manage.

Reserved/Custom CloudFront pricing

The CloudFront pricing page does mention that you can reach out for reserved pricing if you commit to 10TB/month for a year.

Requirements

Must be within the provider's ToS to support file distribution
In searching for options, I've seen a few "Unlimited" or "Unmetered" options, but if you look at the fine print, they're not meant for file storage/distribution, and that's not allowed.
Reasonable file serving redundancy
I'd like to avoid a single machine for file serving. I'll probably use S3 as the "source of truth", but I want to avoid downtime in file serving.
Access logs available
I need to be able to compile basic download stats on files served (apache style logs or similar).

Please Send Feedback/Suggestions!

Again, I'm looking for feedback and suggestions for a cheap, reliable file hosting provider.

Thanks!

Pike Place Market

Tue, 15 Mar 2016 16:47:00 -0700

I just finished my final project for Black and White Photo II at PCNW.

I chose to focus on Pike Place Market here in Seattle.

I spent quite a bit of time there! It was really great to get to know the space, people, and community so well.

The project was a ton of work, and I still feel like I could have used another month printing and editing. While I feel like I will continue to work on and develop this project, I'm really happy at where it ended up and was proud to present these images to the rest of my class.

These were all shot on an older Rolleicord, Kodak TMax 400 film, and printed on Ilford Warmtone Glossy paper.

Pike Place Market

AWS Podcast

Tue, 15 Mar 2016 16:16:00 -0700

I recently started a new weekly aws/cloud/tech podcast with Donnie Flood called Engineers & Coffee.

It's been a lot of fun doing it and I'm really proud of what we've put out there. I can't wait to see how it goes from here.

If you haven't yet, please give it a listen, and please send us some feedback!

Holga 120 Wide Pinhole Camera Review

Thu, 28 Jan 2016 22:21:00 -0700

Thanks to some really amazing Seattle photographers, I've recently started shooting pinhole (photography without a lens!)

While you don't need to buy a camera to shoot pinhole, I think the Holga 120 Wide Pinhole Camera has some nice features and is worth checking out.

I put together my first YouTube photography video with a quick review and intro for the camera.

Please let me know what you think, and subscribe to my channel if you'd like to see more.

You can see some of my pinhole photography on my portfolio.

Thanks!

New Photo Portfolio Site

Sat, 09 Jan 2016 16:37:00 -0700

I still post new photos to my account on Flickr, but I've been wanting to put together a site with an organized, curated set of my favorite photos.

I finally got around to it: photos.ogrodnek.com

I wanted it to be clean, simple, look good on mobile, and FAST, and I think I'm off to a pretty good start.

tech

I went back and forth between building my own thing and using a portfolio site, and of course I ended up building my own thing.

I used Cactus to do the templating and generate the static site content, which is deployed to S3 and uses CloudFront as the CDN.

You can see the source and scripts that ends up generating the site here: https://github.com/ogrodnek/photos.ogrodnek.com

Cactus

So far I like it... I've tried a bunch of other static site generators and I think Cactus fits with me the best so far. Unfortunately, it's not that well documented.

CloudFront

I love CloudFront! It really makes serving content like this so much faster. The new support for gzip compression is a huge help. For personal sites like this, it's also pretty easy to fit comfortably within the free usage tier.

I don't yet have a good/automated setup for doing cache invalidaton yet, or for setting cache expiration headers. I think the newer version of cactus does have a plugin for doing asset fingerprinting to generate unique infinitely cachable names, which would definitely help.

args4j-helpers

Wed, 26 Feb 2014 12:00:00 -0700

I really like using args4j for command-line parsing in both java, and scala, but I found myself writing the same boilerplately code to parse options, deal with help, deal with parsing issues, etc. args4j-helpers is a project that simplifies parsing with args4j.

It provides typical option parsing error handling:

If help (provided via OptionWithHelp base trait bound to --help and -h) is requested, print usage information to stderr, exit with code 0.
If a required option is missing (required=true), print usage information to stderr, exit with code 1 (unless help was requested).

This typical parsing code:

class Options {
  import org.kohsuke.args4j.Option

  @Option(name="--help", aliases=Array("-h"), usage="show this message")
  var help = false

  @Option(name="--blah", aliases=Array("-b"), usage="some val", metaVar="BLAH")
  var blah: Int = 0
}

def main(args: Array[String]) {
  val options = new Options
  val parser = new CmdLineParser(options)

  try {
    parser.parseArgument(args : _*)

    if (options.help) {
      parser.printUsage(System.err)
      sys.exit(0)
    }
  } catch {
    case e: CmdLineException => {
      System.err.println(e.getMessage)
      parser.printUsage(System.err)
      sys.exit(1)
    }
  }
}

can be simplified as:

class Options extends OptionsWithHelp {
  import org.kohsuke.args4j.Option

  @Option(name="--blah", aliases=Array("-b"), usage="some val", metaVar="BLAH")
  var blah: Int = 0
}

def main(args: Array[String]) {
  val options = optionsOrExit(args, new Options)
}

Additionally, the helper class handles the case where help is requested and required arguments are missing (which is missing from the simplified boilerplate code).

The code is in scala, and available on github.

In the future, I'd like to extend it to add better support for more scala-ish types (Option, Seq, etc., which should be mostly possible by implementing additional args4j OptionHandlers).

Crucible Survivor - a code review dashboard

Fri, 14 Feb 2014 10:15:00 -0700

Crucible Survivor Dashboard

I've talked a lot about the importance of code reviews and developer feedback

Crucible Survivor is a Hall of Fame / Hall of Shame dashboard that helps to encourage completing your reviews.

It integrates with with Crucible (obviously) for code review stats.

Scoring

The scoring is pretty simple right now. Each reviewer gets a Fame point for completing a review, and a Shame point for each review they have not yet completed.

We've had this running for a few months now, and the general thinking is it would be better to have open reviews be more shameful the longer they have been kept open. Maybe a future improvement.

Design / Credits

The design is taken from Jira Survivor, which itself was forked from Github Survivor.

Code and Application Architecture

The code is a large departure from the original projects. The original projects use MongoDB to store data scraped from the github/jira APIs, and a python web app to serve the site.

Crucible Survivor is an angular app that is mostly static. The review stats are included in the app as a JS include. The app is hosted as a static website in S3, with the contents of the stats JS generated periodically by a jenkins server.

This was a hack-day experiment in 'static' dashboard apps, and I'm really happy with how it came out. I really like how gathering the content to display is decoupled from the display. Serving the site requires no server infrastructure, and the update process can be very flexible. Finally, it's incredible easy to test/run locally -- just generate a fake stats file and open the site.

Grab the code, get it running!

The code is open sourced, and available on github along with instructions on configuring, running, developing.

I'd love to hear any feedback or comments.

Reader Driven Development

Fri, 15 Feb 2013 12:00:00 -0700

In this talk on Effective ML, Yaron Minsky talks about Reader Driven Development. That is, writing your code with the reader in mind. Making decisions that will make the code more easily read and understood by other developers down the line.

The interest of the reader always pushes in the direction of clarity, simplicity, and the ability to change the code later. In most real projects, code is read and changed many more times than it is written. The readers interest are paramount in that regard.

When writing code the interests of the reader and writer may be at odds, and when faced with a decision, always err in the direction of the reader. The reader is always right. Regardless of team size, it's helpful to program this way. Even code you've written yourself may not be as clear 6 months or a year later otherwise. Great perspective, and I think it fits in nicely with previous posts here on programming style and code reviews (tend to agree with your reviewers, they are the audience!).

Manly Burger @ Unami Burger

Sun, 15 Jul 2012 12:42:00 -0700

The manly burger at unami burger: "beer-cheddar cheese, smoke salt-onion strings, bacon lardons." What else do I have to say?

This might be the best burger I have ever eaten. It's that good.

We also got the beet salad (in the background), and some thin fries. The ketchup is made there and had an interesting flavor - very vinegary, maybe a little truffle oil?

Roast Pork @ Tommy DiNic's

Sun, 08 Jul 2012 12:00:00 -0700

We were just in Philadelphia, and stopped by the Reading Terminal Market in search of the roast pork sandwich at Tommy DiNic’s. You may have seen it featured on Man Vs. Food.

What a sandwich! This is definitely one worth going out of your way for. The official way to get it is with sharp provolone and greens (broccoli rabe). They were out of the rabe, so we had to go with spinach, which is prepared the same way.

The pork is incredibly juicy and delicious, then you have the nice bite of really good sharp provolone, and the garlicky bitterness of the greens. Delicious!

The golden rule of programming style

Wed, 13 Jun 2012 12:00:00 -0700

There's an interesting page on the subject of compilation units per file over at the scala style guide.

The guideline, is, delightfully vague, which I will paraphrase as: Mostly use single files, unless you can't, or unless it's better if you don't.

The author(s) go on to expand on the reasoning behind breaking the guideline:

Another case is when multiple classes logically form a single, cohesive group, sharing concepts to the point where maintenance is greatly served by containing them within a single file. These situations are harder to predict… Generally speaking, if it is easier to perform long-term maintenance and development on several units in a single file rather than spread across multiple, then such an organizational strategy should be preferred for these classes.

This touches on what I consider to be the golden rule of programming style: Make your intent clear and the code easy to read.

Software spends most of its life in maintenance, which is why we have style guides and coding standards. It's valuable to have consistent looking code to promote a shared vocabulary, improve readability, and steer away from confusing or error-prone constructs.

It is just as important to be able to understand, both as an author and as a reviewer, that in certain cases following the letter of the law goes against the main goal of improving readability and maintenance. A one-size-fits-all rule does not always work, and as the authors of this particular guideline mention, "these situations are harder to predict."

Make your intent clear and the code easy to read.

On Code Reviews and Developer Feedback

Tue, 13 Mar 2012 12:00:00 -0700

While he was making his points on stage, I was taking an inventory of the things I didn’t agree with. And when presented with an opportunity to speak with him, I quickly pushed back at some of his ideas. I must have seemed like such an asshole.

His response changed my life. It was a simple thing. He said “Man, give it five minutes.” I asked him what he meant by that? He said, it’s fine to disagree, it’s fine to push back, it’s great to have strong opinions and beliefs, but give my ideas some time to set in before you’re sure you want to argue against them. “Five minutes” represented “think”, not react. He was totally right. I came into the discussion looking to prove something, not learn something.

There’s also a difference between asking questions and pushing back. Pushing back means you already think you know. Asking questions means you want to know. Ask more questions.

This is such a great outlook and a great way to approach the discussion of feedback for code reviews and design reviews.

It's surprising how little time development teams devote to training, or even internal discussion on effective feedback. As developers, we are constantly engaged in this kind of communication: white-boarding sessions, spec reviews, design reviews, code reviews. We're expected to give and receive feedback on a daily basis, but few of us are properly prepared for it. Not only do we lack the training, but we have many negative examples to draw from. Who hasn't been a part of a design review where tempers flare? Properly giving feedback is something that requires constant attention and practice. Receiving feedback can be just as difficult.

Culture of Communication

One of the major pillars of our engineering culture at bizo is "the 3 Cs": Communication, Communication, Communication.

We've tried hard to build a team of engineers that are eager to receive feedback, humble about their abilities, objective and gracious with their feedback, and freely giving of their own knowledge and experience. We see communication as a prerequisite for building a world-class team and developing high-quality code. You often hear the phrase " strong opinions, weakly held," and that is the kind of culture we have tried to build.

Communication is hard. It takes real team agreement and commitment to continued work to keep this culture alive and well. It's important the team views effective communication as important and that the culture supports it.

Code Reviews

Code reviews are something that can easily be approached from the wrong perspective, both as an author or reviewer.

As a reviewer, it can be easy to jump in and argue, to try and push 'your' solution (even though it may be equivalent), to push back instead of asking questions and trying to understand.

As an author, it's far too easy to get attached to your code, to your specific solution/naming/etc. It's also easy to feel like each comment is an attack on your ability, and that by accepting the feedback, this somehow means that you were wrong or did a bad job. Of course, nothing could be further from the truth!

At Bizo, we perform code reviews for every change. They are a major part of our culture of communication. In order to perform effective code reviews, it's important to have some shared guidelines that help support effective communication.

Here are some guidelines we've found to be helpful for performing code reviews:

What is a code review

A careful line-by-line critique of code by peers
Happens in a non-threatening context
Goal is cooperation and mutual learning, not fault finding

Code reviews are a team exercise to improve understanding and make the code better!

When people think of code reviews they usually think of catching bugs. Code reviews do occasionally catch bugs or potential performance problems, but this is rare.

Just as important is fostering a shared understanding of the code and exposure to new approaches, techniques, and patterns. Seeing how your peers program is a great way to learn from them.

Ensuring coding standards and style guides is another way code reviews help. Working on a team it's important to keep readability and quality high using a shared vocabulary.

As an author

As an author, it's important to view each comment as a new opportunity to improve your code. Instead of jumping into defense mode, take a step back and think. Try to approach the code again for the first time with this new perspective. Your team has a lot of experience and varied backgrounds -- draw from them! They are there to help you. Use the gift of their experience and knowledge to improve the code.

Trust the team, and view all comments as action items. Some changes can seem arbitrary, especially when it comes to naming and organization. Unless there's a strong reason, tend to agree with your reviewers. If a reviewer finds something confusing, it is confusing! Code spends most of its life in maintenance and programming is a team sport. Remember that they are your audience, and you want them to be able to understand your code at 4am after a system crash.

As a reviewer

As a reviewer, it's important to take the time to understand the code, think, and ask questions to understand the code before providing feedback. The author probably spent a lot more time thinking about the problem and the approach over the course of the project.

Be strict on coding standard and style guide violations. The real cost of software is maintenance ( 80% according to Sun). It's important the code is easily understood by the team.

Be gentle on personal preferences. If it's not a standard violation and just a matter of personal preference, defer to the author. It's okay to present your perspective, but mention that it's just a preference and not meant to be taken as an action item.

Trust the author. It's often the case that there are many valid approaches to a problem. It's great to present alternative approaches and discuss pros/cons of various approaches. If you see alternative solutions, bring them up! When discussing alternatives, make sure to listen to the author. Remember they are the subject matter expert and you are working together on the same team.

It takes work!

Communication is hard! It's easy to screw-up. It's easy to go into attack or defense mode when you're passionate about what you're doing. It's really something we all need to remind ourselves to work on every day. It's something we need to periodically remind ourselves as a team. Try to view each review as an opportunity to practice these guidelines. Just remember to take a step back, think, and ask questions.

Downtown LA Breakfast Sandwich

Sun, 18 Dec 2011 12:00:00 -0700

Breakfast sandwich from Sandwich Shop in downtown LA.

Egg, cheese, and bacon on ciabatta. The interesting part is that it’s typically served “with vegetables,” which means grilled onions and peppers, jalapeños, and tomato. It’s very tasty and makes me wonder why I don’t see more jalapeños at breakfast.

I’ve been making my own version of this sandwich the last couple weekends and I’ve been really happy with it.

The peppers and onions take awhile to cook, but it’s worth it.

My one regret is that I didn’t get a chance to try out the Bulgogi Cheesesteak at the Sandwich Shop… Next trip.

Cole's Sandwiches in downtown LA

Sat, 17 Dec 2011 12:00:00 -0700

Cole's Sandwiches in downtown LA.

Cole's sandwiches claims to be the originator of the French Dip sandwich. Of course, so does Philippe's.

Either way, it's worth a stop. The sign says "since 1908" and the bar inside looks pretty authentic. It's got a great feel. There's a great draught beer list.

The menu is simple: french dip sandwiches, with your choice of meat. They're served with some ridiculously spicy mustard and an equally spicy pickle that's been pickled alongside some jalapenos. There's something about the spiciness that goes really well with a meaty/bready sandwich. It just works.

I went here for lunch twice while I was in downtown LA. I ate at the bar: beef one day, and pastrami the next.

I can't recommend this place enough. The sandwiches are good, the beers are good, and the place itself has a great feel.

5 minute web framework review - reading params

Tue, 01 Nov 2011 10:00:00 -0700

Through various experiments, hackdays, conversations with other developers, etc. I've found myself experimenting with a few different web frameworks. The focus has been mostly simple webapps / simple REST services written in scala that return html or json. I thought it might be interesting to dive into some focused comparisons in a series of posts.

This is not an exhaustive comparison. I'm going to be focusing on the frameworks I've found the most interesting for my use cases lately, namely scalatra, play, and jersey.

For the first comparison, I want to focusing on reading query and path parameters. Parameter de-serialization has always been a pain. The web uses strings, and strings are messy. Are my required params specified? Are they the right types? Can I easily convert to the types my program expects? Do they pass my validation? etc.

Let's look at how each framework helps us deal with these common concerns.

jersey

I like jersey. It's a reference implementation of JSR-311: Java API for RESTful Web Services. It's also quite nice to work with in scala.

query parameters

With jersey, query parameters are simply specified as method parameters. Simple types are automatically converted, and it's easy to specify defaults. It will also automatically call converters for use with your own complex types. Unfortunately, you must use annotations to map query param names to method params.

def doGet(@QueryParam("name") name: String,
@QueryParam("count") @DefaultValue("2") count: Int): String = {
  "name: %s, count: %d\n".format(name, count)
}

It works pretty much as you'd expect:

$ curl "http://localhost:8080/hello?name=larry&count=5"
name: larry, count: 5

If the types aren't correct, you'll get a 404:

curl "http://localhost:8080/hello?name=larry&count=a" -D -
HTTP/1.1 404 Not Found

path parameters

Path parameters in jersey work pretty much the same way as query params, i.e. typed, with default values and appearing as method arguments. Additionally you can do some simple validation using regular expressions. Their names and path location are specified when defining the route.

@Path("/hello/{userid}")
class Hello {
  def doGet(@PathParam("userid") id: Int)
}

Here's a different example showing some simple regex validation support:

@Path("/hello/{username: [a-zA-Z][a-za-z_0-9] }")

And, again, if the path doesn't match your regex, or type, you will get a 404.

other params

There are also @CookieParam, @HeaderParam annotations for reading cookie and header values, as well as support for pulling in session or request variables using @Context or custom annotations (e.g. I've created @IpAddress for pulling in the ip).

overall thoughts

I really like the automatic de-serialization and type conversion, and having the framework handle errors for incompatible parameters automatically.

I also like the POJO mindset. It's just a function with arguments like any other. All else being equal, this makes testing in any framework super easy.

The annotations do seem a little noisy, especially having to specify the parameter name. I think we can do better.

scalatra

scalatra is also very nice for simple apps, and I've quickly become a fan of scalate which it uses for templating.

When it comes to dealing with parameters though, it feels like a step back. Everything is strings. The fact that it's scala makes it easier to deal with, but it does feel like the framework could go a little further to help you out.

query parameters

To read query parameters, you use the params method from ScalatraServlet. Params is a MultiMapHeadView[String, String]. So yes, you are back to dealing with Strings (or a Seq[String] if using multiParams).

get("/hello") {
  val name:String = params.getOrElse("name", halt(400))
  val count:Int = params.getOrElse("count", "2").toInt

  "name: %s, count: %d\n".format(name, count)
}

Calling this path without a name will generate a 400, as expected:

$ curl "http://localhost:8080/hello" -D -
HTTP/1.1 400 Bad Request

If you don't specify count, you will get the default of 2. However, if you specify a non-int, you'll get a 200 where the contents are the stack trace for the toInt call. Again, your validation is all manual -- if you want better type validation, it's up to you.

$ curl "http://localhost:8080/hello?name=larry&count=a" -D -
HTTP/1.1 200 OK
…


java.lang.NumberFormatException: For input string: "a"

path parameters

Path params work exactly the same way (including being accessed in params), and are named as part of your route

get("/hello/:name/:count") {
  val name:String = params.getOrElse("name", halt(400))
  val count:Int = params.getOrElse("count", "2").toInt

  "name: %s, count: %d\n".format(name, count)
}

$ curl "http://localhost:8080/hello/larry/5"
name: larry, count: 5

overall thoughts

The manual de-serialization seems a little dated, and gets old quick. Scala does make it nicer than it would be in java, since you can do things like params.getOrElse("name", halt(400)), but I would like to see more.

I also miss the POJO mindset… when testing you need to do whatever additional setup is necessary to serialize your params as strings and stick them in a map.

I guess I also don't like that barring convention, there's no formal definition of what parameters you are expecting and what their types are - maybe you are calling params.get somewhere in the middle of your method..

play

play the framework feels a little heavy compared to jersey and scalatra, but if definitely shines when it comes to dealing with parameters.

query parameters

Query parameters in play are done really well. It's almost perfect.

def hello(name: String, count: Int = 2) = {
  "name: %s, count: %d\n".format(name, count)
}

$ curl "http://localhost:9000/hello?name=larry"
name: larry, count: 2

You can even use Option for parameters that may be available:

def hello(name: Option[String], count: Int = 2) = {
  "name: %s, count: %d\n".format(name.getOrElse("anon"), count)
}

One problem is that type conversion errors are silently ignored, and defaults will be used:

$ curl "http://localhost:9000/hello?name=larry&count=a"
name: larry, count: 2

Okay, so they're not really ignored. If you call Validation.hasErrors, it will return true, and you can discover the error. This is the same mechanism you need to use to mark parameter as required:

def hello(name: String, count: Int = 2) = {
  Validation.required("name", name)
  if (Validation.hasErrors) {
  // handle error
  }
}

path parameters

Path parameters work the same way. They're defined with placeholders in your route, and automatically passed in as the correct argument. In play, routes are defined external to your code, in a routes file. E.g.

GET /hello/{name} Application.hello

Our method looks the same as the first Query param example. Calling it looks like this:

$ curl "http://localhost:9000/hello/larry"
name: larry, count: 2

In the case of path parameters, we will get a 404 if missing the parameter (since it won't match our route).

$ curl "http://localhost:9000/hello/" -D -
HTTP/1.1 404 Not Found

overall thoughts

Overall I think parameters in play are done really well.

Like jersey, I really appreciate the POJO approach. play does it even better by eliminating the extra annotations and leveraging scala's default argument support.

Validation does seem a little clunky, though. It seems like more could be done there.

the story of HOD - ahead of its time, obsolete at launch

Mon, 10 Oct 2011 12:00:00 -0700

We've been using AWS as long as bizo has been around, since early 2008. Hadoop has always been a big part of that. When we first started, we were mostly using a shared hadoop cluster. This was kind of a pain for job scheduling, but also was mostly wasteful during off-peak hours... Thus, HOD was born.

From its documentation, "The goal of HOD is to provide an on demand, scalable, sandboxed infrastructure to run Hadoop jobs." Sound familiar? HOD was developed late September and October of 2008, and launched for internal use December 12th, 2008. Amazon announced EMR April of 2009. It's amazing how similar they ended up being... especially since we had no knowledge of EMR at the time. Even though HOD had a few nice features missing from EMR, the writing was on the wall. For new tasks, we wrote them for EMR. We slowly migrated old reports to EMR when they needed changes, or we had the time.

Architecture

HOD borrowed quite liberally from the design of Alexa's GrepTheWeb.

Users submitted job requests to a controller which managed starting exclusive hadoop clusters (master, slaves, hdfs), retrieving job input from S3 to HDFS, executing the job (hadoop map/reduce), monitoring the job, storing job results, and shutting down the cluster on completion. Job information and status was stored in SimpleDB, S3 was used for job inputs and outputs, and SQS was used to manage the job workflow.

Job Definition

Jobs were defined as thrift structures:

struct JobRequest {
   1: JobConf job = {},
   2: i32 requested_nodes = 4, // requested number of hadoop slaves
   3: string node_type = "m1.large", // machine size
   4: i32 requested_priority, // job priority
   5: string hadoop_dist, // hadoop version e.g. "0.18.3"
   6: set depends_on = [], // ids of job dependencies
   7: list on_success = [], // success notification
   8: list on_failure = [], // failure notification
   9: set flags = [] // everyone loves flags
 }

 struct JobConf {
   1: string job_name,
   2: string job_jar,  // s3 jar path
   3: string mapper_class,
   4: string combiner_class,
   5: string reducer_class,
   6: set job_input,  // s3 input paths
   7: string job_output,  // s3 output path (optional)
   8: string input_format_class,
   9: string output_format_class,
   10: string input_key_class,
   11: string input_value_class,
   12: string output_key_class,
   13: string output_value_class,

   // list of files for hadoop distributed cache
   14: set user_data = [],
   15: map other_config = {}, // passed directly JobConf.set(k, v)
}

You'll notice that dependencies could be specified. HOD could hook up the output of 1 or more jobs into the input of a job and wouldn't run the job until all of its dependencies have successfully completed.

User Interaction

We had a user program, similar to emr-client that helped construct and job jobs, e.g.:

JOB="\
 -m com.bizo.blah.SplitUDCMap \
 -r com.bizo.blah.UDCReduce \
 -jar com-bizo-release:blah/blah/blah.jar
 -jobName blah \
 -i com-bizo-data:blah/blah/blah/${MONTH} \
 -outputKeyClass org.apache.hadoop.io.Text \
 -outputValueClass org.apache.hadoop.io.Text \
 -nodes 10 \
 -nodeType c1.medium \
 -dist 0.18.3 \
 -emailSuccess larry@bizo.com \
 -emailFailure larry@bizo.com \
"

$HOD_HOME/bin/hod_submit $JOB $@

There was also some support for querying jobs:

As well as support for viewing job output, logs, counters, etc.

Nice features

We've been very happy users of Amazon's EMR since it launched in 2009. There's nothing better than systems you don't need to support/maintain yourself! And they've been really busy making EMR more easy to use and adding great features. Still, there are a few things I miss from HOD.

Workflow support

As mentioned, HOD had support for constructing job workflows. You could wire up dependencies amount multiple jobs. E.g. here's an example workflow

It would be nice to see something like this in EMR. For really simple workflows, you can sometime squeeze them into a single EMR job as multiple steps, but that doesn't always make sense and isn't always convenient.

Notification support

HOD supported notifications directly. Initially just email notifications, but there was a plugin structure in place with an eye towards supporting HTTP endpoint, and SQS notifications.Yes, this is possible by adding a custom EMR job step at the end that checks the status of itself and sends an email/failure notification... But, c'mon, why not just build in easy SNS support? Please?

Counter support

Building on that, HOD had direct support/understanding for hadoop counters. When processing large volumes of data, they become really critical in tracking the health of your reports over time. This is something I really miss. Although, it's less obvious how to fold this in with Hive jobs, which is how most of our reports are written these days.

Arbitrary Hadoop versions

HOD operated with straight hadoop, so it was possible to have it install an arbitrary version/package just by pointing it to the right distribution in S3.Since Amazon isn't directly using a distribution from the hadoop/hive teams, you need to wait for them to apply their patches/changes and can only run with versions they directly support. This has mostly been a problem with Hive, which moves pretty quickly.It would be really great if they could get to a point where their changes have been folded back into the main distribution.Of course, this is probably something you can do yourself, again with a custom job step to install your own version of Hive… Still, they have some nice improvements, and again, it would be nice if it were just a simple option to the job.

The Past / The Future

Of course, HOD wasn't without its problems :). It's become a bear to manage, especially since we pretty much stopped development / maintenance (aside from rebooting it) back in 2009. It was definitely with a sigh of relief that I pulled the plug.

Still, HOD was a really fun project! It was an early project for me at Bizo, and it was really amazing how easy it was to write a program that starts up machines! and gets other programs installed and running! Part of me wonders if there isn't a place for an open source EMR-like infrastructure somewhere? Maybe for private clouds? Maybe for people who want/need more control? Or for cheapskates? :)

Or maybe HOD v2 is just some wrappers around EMR that provides some of the things I miss : workflow support, notifications, easier job configuration...Something to think about for that next hack day :).

hackday - analog meters

Fri, 13 Aug 2010 10:00:00 -0700

For this last hackday, I decided to work on something more hardware hacking related. At this year's Maker Fair, I was really inspired by all the cool stuff people were building, so I picked up an arduino and started playing around with a couple of things.

I've always wanted to have some cool old-school analog VU type meters displaying web requests.

Here's my completed hackday project:

Here's a view of the components from the back:

It's battery operated and receives data wirelessly over RF from another arduino I have hooked up via serial to my laptop.

It's pretty simple, but I'm still totally psyched about how it came out.

The main components are some analog panel meters (kinda pricey, but awesome), and an RF receiver. The frame is a piece of scrap acrylic from TAP Plastics that I drilled and cut to size, and the stand is a piece of a wire clothes hanger bent to shape.

Connected to my computer is a another arduino (actually a volksduino) that receives updates over USB and sends the data out over RF:

You may be asking, why bother with wireless if you need a computer hooked up through serial anyway. Or you may ask why not just connect to a wireless network directly.

Well, I wanted the meters to be able to be moved around, or mounted on a wall... I wanted them wireless. But, it turns out that wireless and even ethernet solutions for connecting an arduino to the internet directly are comparatively pretty expensive. Even using bluetooth is expensive. My long term plan is to have a single arduino connected to the internet directly (via ethernet or wireless), and have it serve as a proxy over RF for the others... So this is a bit of work towards that.

I wrote a bit of Java code to connect to amazon's cloudwatch to pull the load balancer statistics for two of our services. I then discovered it's near impossible to connect to anything over USB in Java... It is ridiculous. Luckily, it's REALLY easy to do this with Processing, so I wrote a simple processing program that used my cloudwatch library and wrote it out to serial.

And that's really it. The arduino reads data over serial, and periodically sends it over RF. The arduino hooked up to the meters simply reads the values over RF and sets the meters to display a scaled version of the results. They're showing requests per second. We get a huge amount of requests per second with these services, so the numbers on the dial aren't actually correct (I need to make some custom faceplates). It also flashes an LED every time it gets a RF transmission.

Here's a quick video of it in action:

The one thing I'm not crazy about is that the maximum resolution you can get from cloudwatch is stats per minute, so the meters don't actually change as often as I would like.

Still, pretty cool. I'm looking forward to building some more displays like this in the future.