Amazon just released the BETA of a really useful service if you’re dealing with huge amounts of data. 

AWS Import/Export accelerates moving large amounts of data into and out of AWS using portable storage devices for transport. AWS transfers your data directly onto and off of storage devices using Amazon’s high-speed internal network and bypassing the Internet. For significant data sets, AWSImport/Export is often faster than Internet transfer and more cost effective than upgrading your connectivity.

How does it work? Continue reading…

Amazon announced today the public beta of Amazon Elastic MapReduce, a web service that enables businesses, researchers, data analysts, and developers to easily and cost-effectively process vast amounts of data. It utilizes a hosted Hadoop framework running on the web-scale infrastructure of Amazon Elastic Compute Cloud (Amazon EC2) and Amazon Simple Storage Service (Amazon S3).

Using Amazon Elastic MapReduce, you can instantly provision as much or as little capacity as you like to perform data-intensive tasks for applications such as web indexing, data mining, log file analysis, machine learning, financial analysis, scientific simulation, and bioinformatics research. Amazon Elastic MapReduce lets you focus on crunching or analyzing your data without having to worry about time-consuming set-up, management or tuning of Hadoop clusters or the compute capacity upon which they sit. Continue reading…

Feeling the pressure from Microsoft’s Windows Azure, Amazon works hard on releasing GUI tools for managing their services. Until now, the only GUI way for managing services were a couple of Firefox plugins - Elasticfox for managing EC2 and S3fFox for organizing S3 and CloudFront. However, Amazon just released, in BETA of course, its brand new AWS Management Console, (using Yahoo’s YUI framework, using JSP in the backend).

The initial release of the AWS Management Console provides a graphical user interface for Amazon EC2, with additional Amazon infrastructure services scheduled to be added to the console in the coming months. You can create Elastic Block Store volumes and assign Elastic IP’s to your instances.

The features planned for the future releases look promising: Continue reading…

In a story at Wall Street & Technology, Penny Crosman writes about how Nasdaq stock exchange looks to the clouds for data storage. 

Nasdaq stores lots of terabytes of Nasdaq, NYSE and Amex data using Amazon Simple Storage Service. Nasdaq adds 30 gigabytes to 80 gigabytes of data every day to S3, about 300,000 flat files each representing 10 minutes’ worth of trading activity on a stock. The data retrieval time, is less than one second, and the system scales instantly.

They are using S3 for a new product called Market Replay that enables brokerage firms to show customers and regulators that best-execution requirements were met for a given trade. The application was developed using Adobe Flex.

Nasdaq plans to develop future applications in Amazon’s Elastic Compute Cloud, according to Claude Courbois, associate VP, product development, Nasdaq Data Products. Continue reading…

Cloudfork AWS is a new open source project that provides easy access from Smalltalk to the Amazon Web Services that are related to cloud computing. The following services are supported: SimpleDB, SQS, S3.

The code is hosted on Squeaksource. The plan is to port Cloudfork AWS to other Smalltalk dialects as soon as the code is reasonably stable. The goal of the Cloudfork AWS project is simply to make the API’s easily usable from Smalltalk. 

Here’s the official blog of the project.

Amazon web services blog has information about the latest and most powerful feature of S3 - Requester Pays which works at the level of an S3 bucket. If the bucket’s owner flags it as Requester Pays, then all data transfer and request costs are paid by the party accessing the data.

Like any other Amazon Service, there’s cost associated with it. By marking a bucket as Requester Pays, data owners can provide access to large data sets without incurring charges for data transfer or requests. For example, they could make available a 1 GB dataset at a cost of just 15 cents per month. Requesters use signed and specially flagged requests to identify themselves to AWS, paying for S3 GET requests and data transfer at the usual rates — 17 cents per GB for data transfer and 1 cent for every 10,000 GET requests. Here’s the documentation explaining in details how the feature should and can be used. 

Amazon is also pushing Amazon DevPay service to developers, they have an integration of the Requester Pay model with DevPay described here.

Panda is an open source solution for video uploading, encoding and streaming, running completely within Amazon’s Web Services using customized EC2 instances, S3 and SimpleDB. It has support for the encoding profiles which FFmpeg supports. They include FLV for flash and H264 for iPhone.

The service is easy to integrate with your application. The EC2 instance will provide a simple REST (both YAML and XML formats support) API for listing, creating, editing and deleting videos. When a new video is created on your site the actual file upload takes place in a popup or iframe. Doing so means that the large video file is uploaded directly to your Panda EC2 instance so you don’t have to handle it within your application. The server also is configured to support an upload progress bar so user’s can see the video upload in progress. It cannot get any easier than this.

The range of encoding support does not depend on Panda, but rather depends on ffmpeg and Libavcodec(open source encoder/decoder tools and libraries Panda use underneath) encoding capability. Wikipedia page has a list of implemented video codecs. For Panda AMI setup, see this and this google group thread. Continue reading…

From Palo Alto Software, creators of Business Plan Pro and Marketing Plan Pro, here’s the first software as a service initiative: Email Center Pro. Simply put, Email Center Pro is a a collaborative email solution for the small business market.

Email Center Pro makes it easy to manage customer email sent to your info@, sales@, or help@ addresses. It saves you time, increases your customer satisfaction, and lowers your operating costs. How? Mainly, the service allows you to manage all of your email queues from one location and efficiently route emails from one mailbox to another. You can easily assign any email to your staff for a prompt response.

Email Center Pro Overview

It sounds like a simple service, and it really is. Like any service should be. It is easy to use and feature reach: you can add internal notes to any email conversation, you can tag incoming or outgoing mail to categorize messages for fast and easy retrieval. You can search your emails(duh!) but most importantly you can save a search, so future emails will be already categorized by your search.

One of the most useful feature we found to be the alerts. You can get alerts about almost anything at any time. Also, The UI is well done, really usable. Continue reading…

The tool provides a visual interface to Amazon SimpleDB in the form of a Firefox plugin for querying and updating your Simple DB database domains.

This new tool complements the other visual tools available for managing Amazon services, also provided as Firefox plugin: Elastic Fox - the extension for interacting with Amazon EC2 and S3Fox - Firefox organizer for S3 and CloudFront, helps you organize, manage, and store your files on Amazon S3 and distribute them through CloudFront.

The tool is developed by Bizo, a B2B advertising network running entirely on top of AWS. The code is open source an hosted on github. Read more on the Bizo blog.

CloudFront, the new service from Amazon, will provide you with a high performance way to distribute popular, publicly readable content to your customers all over the world, with low latency and high data transfer rates.

CloudFront was designed to meet the following goals:

• Allow developers and businesses to get started easily, with no dollar or volume commitments. Like other services, this one will be pay-as-you-go.
• Be simple and easy to use. In fact, a single API call is all that’s needed for you to start delivering your content.
• Work seamlessly with Amazon S3, for durable storage of the definitive versions of your content.
• Have a global presence, using edge locations on three continents in order to deliver your content from the most appropriate location

Like all Amazon other services, CloudFront was designed with ease of use in mind from the very beginning.  There are no minimum usage commitments, no monthly fees, and no need to even talk to Amazon. Here’s what you do: Continue reading…