During the last few years I have been implementing AWS serverless architectures for my clients. So far I have enjoyed it very much. That’s why I’ve decided to share some of my experiences in a series of blog posts. In this first post, I will explain “high level” what AWS serverless means to me and what the key concepts of serverless development are.
System of Services
People often ask me: “What do you mean with serverless development?“. High level, I would describe it as building a “system of services” (at least that is what serverless development feels like to me).
Allow me to explain…
The definition of a system, according to Wikipedia (systems theory):
A system is a group of interacting or interrelated elements that act according to a set of rules to form a unified whole.
Amazon Web Services (or in short AWS) enables you to build such a system.
In the cloud.
AWS is a collection of services.
Services are reusable elements that you can configure and link together, to form a unified whole, your application. Every service fulfills a certain functionality or purpose, and adds something useful to the whole. Services are the building blocks of your application.
In AWS, services can interact with each other in an event-driven way: the services act upon state changes of other services and can trigger one another. Interaction between services is what makes your application alive.
AWS Services
AWS offers a lot of services that can be used for implementing common use cases. So you don’t have to write everything yourself. These services cover all kinds of interesting areas: databases, storage, computing, analytics, security, … but also exotic ones, like Alexa, Internet of Things, Machine Learning, robotics…
The list grows year after year.
AWS offers so many services that it is impossible to fully know (and keep up with) all of them. Eventually every developer or architect will choose their favorites for their toolbox. These are some of my favorites, for building serverless architectures:
-
AWS LambdaExposes little pieces of “your own” code as a service. One of the core services in AWS serverless.
-
Amazon API Gateway
A service that can be placed in front of other services, to expose them as web services (REST and Web Sockets).
-
Amazon Simple Storage Service (S3)
A service that provides object storage in the cloud.
-
Amazon DynamoDB
A fully managed NoSQL database service that supports key-value and document data structures.
-
Amazon Simple Queue Service (SQS)
A service for implementing message queues. This service can be used for decoupling parts of your applications and/or for enforcing flow control. A publisher can “push” messages on a queue. Later, other components can then “pull” the messages from the queue, for processing.
-
Amazon Simple Notification Service (SNS)
A service that allows implementing publish/subscribe messaging patterns. By using Amazon SNS topics, your publisher systems can filter and fan out messages to a large number of subscriber systems (“push” based).
-
AWS Step Functions
A service for orchestrating other services, as a state machine.
As you can see, every service in AWS has its own distinctive icon, which makes it very easy to use in architecture diagrams…
Functions as a Service
One of the most important services in AWS Serverless is the AWS Lambda service.
Remember lambdas? Those small reusable pieces of code?
While lambdas did not exist yet in the first versions of the Java programming language (they were introduced in 2014 with Java 8), they have existed already for a very long time in functional programming and scripting languages, like Python, JavaScript, …
Some examples:
Java
Operation square = (x) -> {
return x * x;
};
Python
square = lambda x : x * x
JavaScript
const square = (x) => {
return x * x
}
Scheme
(lambda (x) (* x x))
In fact, the origin of the lambda (λ) is actually much older… Long before computers existed, lambdas already existed in mathematics. They were invented by a guy called Alonzo Church. He described them in a document called the “Lambda Calculus“, in 1930 (!).
In Lambda Calculus, you would write lambda functions, represented as mathematical functions. Numbers were also represented as a lambda function. To solve the function, you’ve had to apply recursive substitution (iteratively resolving parts of your function into smaller functions), until the solution appeared (as a function that could no longer be resolved). With lambda calculus, mathematicians were able to solve complex problems that could not have been solved before.
Six years later, machines were built that could solve these functions for you (the so-called Turing machines, named after its inventor Alan Turing). And so, the first computers were born. Lambdas played a big role in the history and evolution of computers, and — as we will see now — they still play a big role in the innovation of modern technologies.
AWS has taken the good old lambdas to the next level…
With the AWS Lambda service, you can deploy small pieces of “your own” code, as services, in the cloud. AWS takes care of scaling of instances, high availability, logging, disaster recovery. The language that you use to express yourself isn’t even important, since AWS Lambda supports many programming/scripting languages (Java, NodeJS, Python, …).
In serverless terminology, this is also called Functions as a Service (or FaaS).
Amazon was the first provider offering FaaS to the public, followed by Google and Microsoft (Azure).
Event Driven Architecture
A system needs to have interaction between its elements, to be alive.
In AWS, services can interact with each other in an event-driven way:
An event happening in an event source (can be a service, data or an external resource), e.g. storing a file on S3, inserting a record in a DynamoDB database, sending a request to an API Gateway REST endpoint, … can trigger invocation of a service. The service will receive the details of the event and will act upon it.
What services are triggered, by which event, and in which conditions, is all defined by configuration.
To make it easy to deploy and configure services (DevOps), AWS offers an Infrastructure as Code solution called CloudFormation, but you can also use other tools like Terraform or the Serverless Framework.
Serverless FaaS
But what makes an AWS Lambda truly “serverless”? And what is the difference with, for example, a Spring Boot application?
Serverless FaaS offers some additional advantages compared to traditional FaaS…
- No servers to provision or manage
In pure serverless, you will never have to think about hardware or operating systems again… You just write your code, deploy it, and link services together via configuration. AWS will take care of the rest. - Scales with usage
Services will automatically spawn and scale with usage. If load increases, the number of instances will increase, by some clever (configurable) algorithm. - Built-in high availability and disaster recovery
AWS offers built-in high availability and disaster recovery, out of the box. - Never pay for idle again
You only pay for what you use… You will never pay for idle time since you don’t have servers. AWS does all the heavy lifting for you, so you can concentrate on what really matters: your business logic.
Developer Experiences
In my experience…
The difficulty of AWS Serverless development is not in writing the business logic, nor in configuring the services. Lambdas tend to be simple, small, pieces of code. The difficulty is in choosing the right architecture. As there can be multiple solutions to the same problem, you need to know which services are the best fit for your use case and what the life cycle, limitations, pricing, best practices of those services are… So take your time to get to know your services…
Lambdas are usually implementations of the same patterns over and over again (like doing transformations, applying some business logic, doing some orchestration, …). Try to keep it simple. Don’t use too many frameworks. Some languages (like NodeJS and Python) already offer a lot of tools out of the box, for doing transformations.
At first, AWS might look a little bit overwhelming and it might be difficult to find your way. I would suggest that you start playing around with the AWS console first (create some lambdas, connect it to an API gateway, …) before doing Infrastructure as Code. Just research how the services can be configured and what is available. You will learn a lot in a short amount of time.
AWS services are also very well documented, look for example at the Lambda documentation and developer guide. You can also find a lot of very good tutorials about it on the internet and youtube.
Architecture Example
Let me conclude with a small example, to give you a better idea of what a serverless architecture might look like:
This is an application that I have once built during a (dutch) presentation on AWS Serverless. In this example, we expose a REST API for storing and processing data (a pattern that is commonly implemented at my clients).
The API Gateway exposes the following resources:
- GET /info
Returns some information about the application. - POST /data
Stores data in a S3 bucket and triggers processing. - GET /data/{id}
Allows you to retrieve data from the S3 bucket.
Every request to one of these resources, will be handled by its own lambda:
- Info
- Receiver
- Downloader
The “Receiver” lambda will implement some simple code for storing the data in a S3 bucket. When new data is placed in the S3 bucket, the “Processor” lambda will be triggered (asynchronously, in an event driven way).
The source code of this application can be found here. If you are searching for some more architecture examples, AWS is also offering some reference architectures here.
Conclusion
The main things to learn from this blog post are:
- AWS services are the building blocks of an AWS Serverless application
- Services can be linked together in an Event Driven Architecture, by configuration.
- The AWS Lambda service allows you to expose “small pieces of your code” as services (also called Functions as a Service)
- AWS Lambdas are supposed to be simple, small pieces of code
- Don’t use too many frameworks when writing AWS Lambdas, keep it simple
- The advantages of Serverless FaaS are:
– No servers to provision or manage
– Scales with usage
– Built-in high availability and disaster recovery
– Never pay for idle
In my next blog posts, I will explain the details of some of my favorite services, with some code examples.
Stefan Bangels
Software Craftsman
Stefan is a passionate and very driven Software Craftsman. He has over 19 years of experience in the various domains of the IT world. He currently mainly takes on tasks as senior developer, team lead and architect. His dedication and responsibility ensure that every assignment is in good hands with him. He is very eager to learn and continuously trains himself to keep up with the latest technology. He is also co-responsible for coaching other Software Crafters within Continuum and has already given various internal and external training courses.
