Claudio Cherubino's blog Life of a Googler


Using JetS3t to interact with Google Storage

Google announced a lot of cool stuff at Google I/O, including Google TV, Android Froyo and App Engine for Business but it was another project that caught my attention and it is Google Storage for Developers.

Cloud storage is a huge business and many successful online services such as Dropbox, SlideShare or Twitter already use Amazon S3 for their storage needs. The idea is simple: you store your files on Google's infrastructure and access them from anywhere, paying only for what you use.

Google Storage is a RESTful service so it is possible to interact with it through the standard HTTP methods (GET, POST, PUT and DELETE) as explained in the Developer's Guide but it is much easier to rely on the client libraries which hide the complexities of the raw protocol.
A Python library can be downloaded from the project website but Google Storage also understands Amazon S3 protocol so that any existing tool for S3 can work with Google Storage with minor (if any) modifications.

Among all the toolkits for Amazon S3, JetS3t is probably the best known and most widely adopted. It is an open-source Java suite that officially added support for Google Storage in the recently released version 0.8.0.

JetS3t is released under the Apache License 2.0, so it is free for commercial or non-commercial use and can be modified as needed. Its source code can be downloaded from the project page on BitBucket where you can also find the instructions on how to build it or contribute to the project.

Using JetS3t to write a Java application that interacts with Google Storage is very easy. It is possible to connect to the service, create buckets and upload/download objects with less than 10 lines of code:

// Google Storage login credentials are required to manage GS accouponunts.
GSCredentials credentials = new GSCredentials(ACCESS_KEY, SECRET_KEY);

// To communicate with Google Storage use the GoogleStorageService.
GoogleStorageService service = new GoogleStorageService(credentials);

// To store data in GS you must first create a bucket, a container for objects.
GSBucket bucket = service.createBucket("jets3t-test-bucket");

// Create a GSObject based on a local file
File localFile = new File("my_file.txt");
GSObject localObject = new GSObject(localFile);

// Upload the object to our test bucket in GS.
localObject = service.putObject("jets3t-test-bucket", localObject);

// Download the data object we just uploaded
GSObject remoteObject = service.getObject("jets3t-test-bucket", "my_file.txt");

The GoogleStorageService exposes all the methods to manipulate the cloud filesystem and only takes a set of credentials to be instantiated. Besides the basic set of commands presented in the code snippet above, there are other methods to manage Access Control Lists or to copy/move/rename objects.
The source code distribution also includes a complete code sample that covers all the functionalities described in the Google Storage Developer's Guide.

Are you already thinking about the next big thing that can be built on top of this service?