On March 18, 2021, Amazon Web Services (AWS) introduced a new feature that helps customers use the same dataset across multiple applications, regardless of their different requirements. The new S3 Object Lambda feature retrieves data from S3 and then automatically processes it and brings changes to work with existing applications.
AWS’s Lambda serverless function service was a thing to give attention to at the last year’s AWS conference.
There is a list of scenarios where the same dataset will be helpful for different applications with different data demands. For example, some e-commerce sites may want to use rich details from the customer dataset to serve marketing campaigns, but it may want to exclude some details for analytics purposes.
In a typical scenario, a customer would have to make copies of the dataset for every application or require managing or building infrastructure as a proxy layer to do the processing appropriately.
With S3 Object Lambda, customers can add their own code to process data retrieved from S3 before getting back to the application. The Lambda function is called out inline with a standard S3 GET request.
This ability will prove to be extremely useful for a wide range of cases like redacting personally identified information, augmenting data, converting data formats or compressing or decompressing files as they are downloaded.