AWS re:Invent: The synchronous stack - AWS Data Exchange for APIs
As AWS re:Invent 2021 swings into gear, we’re looking for software application development.
More specifically, we’re looking for cloud software application development… and even more specifically than that, we’re looking for cloud developer updates that drop at the sharpest end of deployment tangibility for the types of real world use cases we are seeing in the web-scale world of what everybody now likes to call ‘modern cloud’ computing.
Clearly, among the sharpest tools (okay, perhaps rope bindings rather than blades) in the box are Application Programming Interfaces (APIs).
Even before the day #1 keynote session was staged, AWS developer advocate Alex Casalboni detailed some of his division’s work related to API-based integrations.
First, let’s recap on AWS Data Exchange… this service enables engineers to discover, subscribe to and use file-based datasets via Amazon Simple Storage Service (Amazon S3) offered by third parties such as Reuters, Foursquare, IMDb etc.
There is also a data ingestion function here i.e. AWS Data Exchange for Amazon Redshift is designed to makes it easier to ingest third-party data in an Amazon Redshift data warehouse, without any manual processing or transformation.
From static to synchronous
All of which is great for static datasets, but today of course we know that many applications will require frequent and synchronous retrieval of data, even if those elements of data are comparatively small.
That could be a market price update, an exchange rate, a social media feed monitor/tracker… anything, basically.
AWS developer advocate Casalboni: The static stack now shimmies over to continuous & frequent synchronous reality.
Casalboni reminds us that data APIs enable cloud engineers to answer specific questions quickly without having to build ad-hoc data pipelines to ingest, process and analyse bulk datasets. But each API provider has its own ease of use, SDK, documentation and authentication mechanisms, which makes this harder than it needs to be.
“Today, I’m happy to announce the general availability of AWS Data Exchange for APIs, a new capability that lets you find, subscribe to, and use third-party APIs with consistent access using AWS SDKs, as well as consistent AWS-native authentication and governance. This simplifies the lives of developers and IT administrators who have to integrate and secure the access to multiple third-party APIs,” explained Casalboni, in an AWS developer blog.
What this means then is that developers can make RESTful or GraphQL API calls directly to AWS Data Exchange and receive synchronous responses that contain the information they need.
“Using the AWS SDK in the programming language of your choice. We take care of integrating with the API provider, implementing proper authentication, managing the API subscription, and ensuring charges appear on your AWS bill. You can manage API access centrally with AWS Identity and Access Management (IAM),” said Casalboni.
His message to a data provider is that they can now ake their API discoverable to what is millions of AWS customers by listing it in the AWS Data Exchange catalogue using an OpenAPI specification and fronting it with an Amazon API Gateway endpoint.
AWS Data Exchange for APIs is generally available now in all AWS Regions where AWS Data Exchange is available and developers can check out new AWS Data Exchange for APIs documentation to learn more.