Microservices are defined as a self-regulating, and independent codebase that can be written and maintained even by a small team of developers. Microservices Architecture consists of such loosely coupled services with each service responsible for the execution of its associated business logic.
The services are separated from each other based on the nature of their domains and belong to a mini-microservice pool. Enterprise mobile app developers leverage the capabilities of this architecture especially for complex applications.
Microservices Architecture allows developers to release versions of software thanks to sophisticated automation of software building, testing, and deployment – something that acts as a prime differentiation point between Microservices and Monolithic architecture.
Benefits
Since the services are bifurcated into pools, the architecture design pattern makes the system highly fault-tolerant. In other words, the whole software won’t collapse on its head even if some microservices cease to function.
An enterprise mobile app development company working on such an architecture for clients can deploy multiple programming languages to build different microservices for their specific purpose. Therefore the technology stack can be kept updated with the latest upgrades in computing.
This architecture is a perfect fit for applications that need to scale. Since the services are already independent of each other, they can scale individually rather than overloading the entire system with the need to expand.
Services can be integrated into any application depending upon the scope of work.
Potential Drawbacks
Since each service is unique in its ability to contribute to the whole codebase, it could be challenging for an enterprise mobile application development company to interlink all and operate so many distinctive services seamlessly.
Developers must define a standard protocol for all services to adhere to. It is important to do so, as the decentralized approach towards coding microservices in multiple languages can pose serious issues while debugging.
Each microservice with its limited environment is responsible to maintain the integrity of the data. It is up to the architects of such a system to come up with a universally consistent data integrity protocol, wherever possible.
You definitely need the best of breed professionals to design such a system for you as the technology stack keeps changing.
Ideal For
Use Microservices Architecture for apps in which a specific segment will be used heavily than the others and would need a sporadic burst of scaling. Instead of a standalone application you may also deploy this for a service that provides functionality to other applications of the system.
Ever wondered how large enterprise scale systems are designed? Before major software development starts, we have to choose a suitable architecture that will provide us with the desired functionality and quality attributes. Hence, we should understand different architectures, before applying them to our design.
What is an Architectural Pattern?
According to Wikipedia,
An architectural pattern is a general, reusable solution to a commonly occurring problem in software architecture within a given context. Architectural patterns are similar to software design pattern but have a broader scope.
In this article, I will be briefly explaining the following 10 common architectural patterns with their usage, pros and cons.
Layered pattern
Client-server pattern
Master-slave pattern
Pipe-filter pattern
Broker pattern
Peer-to-peer pattern
Event-bus pattern
Model-view-controller pattern
Blackboard pattern
Interpreter pattern
1. Layered pattern
This pattern can be used to structure programs that can be decomposed into groups of subtasks, each of which is at a particular level of abstraction. Each layer provides services to the next higher layer.
The most commonly found 4 layers of a general information system are as follows.
Presentation layer (also known as UI layer)
Application layer (also known as service layer)
Business logic layer (also known as domain layer)
Data access layer (also known as persistence layer)
Usage
General desktop applications.
E commerce web applications.
Layered pattern
2. Client-server pattern
This pattern consists of two parties; a server and multiple clients. The server component will provide services to multiple client components. Clients request services from the server and the server provides relevant services to those clients. Furthermore, the server continues to listen to client requests.
Usage
Online applications such as email, document sharing and banking.
Client-server pattern
3. Master-slave pattern
This pattern consists of two parties; master and slaves. The master component distributes the work among identical slave components, and computes a final result from the results which the slaves return.
Usage
In database replication, the master database is regarded as the authoritative source, and the slave databases are synchronized to it.
Peripherals connected to a bus in a computer system (master and slave drives).
Master-slave pattern
4. Pipe-filter pattern
This pattern can be used to structure systems which produce and process a stream of data. Each processing step is enclosed within a filter component. Data to be processed is passed through pipes. These pipes can be used for buffering or for synchronization purposes.
Usage
Compilers. The consecutive filters perform lexical analysis, parsing, semantic analysis, and code generation.
Workflows in bioinformatics.
Pipe-filter pattern
5. Broker pattern
This pattern is used to structure distributed systems with decoupled components. These components can interact with each other by remote service invocations. A broker component is responsible for the coordination of communication among components.
Servers publish their capabilities (services and characteristics) to a broker. Clients request a service from the broker, and the broker then redirects the client to a suitable service from its registry.
In this pattern, individual components are known as peers. Peers may function both as a client, requesting services from other peers, and as a server, providing services to other peers. A peer may act as a client or as a server or as both, and it can change its role dynamically with time.
This pattern primarily deals with events and has 4 major components; event source, event listener, channel and event bus. Sources publish messages to particular channels on an event bus. Listeners subscribe to particular channels. Listeners are notified of messages that are published to a channel to which they have subscribed before.
Usage
Android development
Notification services
Event-bus pattern
8. Model-view-controller pattern
This pattern, also known as MVC pattern, divides an interactive application in to 3 parts as,
model — contains the core functionality and data
view — displays the information to the user (more than one view may be defined)
controller — handles the input from the user
This is done to separate internal representations of information from the ways information is presented to, and accepted from, the user. It decouples components and allows efficient code reuse.
Usage
Architecture for World Wide Web applications in major programming languages.
This pattern is useful for problems for which no deterministic solution strategies are known. The blackboard pattern consists of 3 main components.
blackboard — a structured global memory containing objects from the solution space
knowledge source — specialized modules with their own representation
control component — selects, configures and executes modules.
All the components have access to the blackboard. Components may produce new data objects that are added to the blackboard. Components look for particular kinds of data on the blackboard, and may find these by pattern matching with the existing knowledge source.
Usage
Speech recognition
Vehicle identification and tracking
Protein structure identification
Sonar signals interpretation.
Blackboard pattern
10. Interpreter pattern
This pattern is used for designing a component that interprets programs written in a dedicated language. It mainly specifies how to evaluate lines of programs, known as sentences or expressions written in a particular language. The basic idea is to have a class for each symbol of the language.
Usage
Database query languages such as SQL.
Languages used to describe communication protocols.
Interpreter pattern
Comparison of Architectural Patterns
The table given below summarizes the pros and cons of each architectural pattern.
Comparison of Architectural Patterns
Hope you found this article useful. I would love to hear your thoughts. 😇
Not very long ago, with the exponential increase in number of libraries and their dependencies, dependency management was becoming very complex task which required good amount of technical expertise to do it correctly. With the introduction of String boot starter templates, you can get a lot of help in identifying the correct dependencies to use in project if you want to use any popular library into your project.
Spring Boot comes with over 50+ different starter modules, which provide ready-to-use integration libraries for many different frameworks, such as database connections that are both relational and NoSQL, web services, social network integration, monitoring libraries, logging, template rendering, and the list just keeps going on.
How starter template work?
Spring Boot starters are templates that contain a collection of all the relevant transitive dependencies that are needed to start a particular functionality. Each starter has a special file, which contains the list of all the provided dependencies Spring provides.
These files can be found inside pom.xml files in respective starter module. e.g.
The spring-boot-starter-data-jpa starter pom file can be found at github.
This tells us that by including spring-boot-starter-data-jpa in our build as a dependency, we will automatically get spring-orm, hibernate-entity-manager and spring-data-jpa. These libraries will provide us all basic things to start writing JPA/DAO code .
So next time when you want to give your project any specific functionality, I will suggest to check for existing starter templates to see if you can use it directly. Ongoing community additions are always on, so this list is already growing and you can contribute to it as well.
Popular templates and their transitive dependencies
I am listing down some very frequently use spring starters and what dependencies they bring along, for information only.
Spring Boot is a Spring framework module which provides RAD (Rapid Application Development) feature to the Spring framework. It is highly dependent on the starter templates feature which is very powerful and works flawlessly.
Spring boot modules
1. What is starter template?
Spring Boot starters are templates that contain a collection of all the relevant transitive dependencies that are needed to start a particular functionality. For example, If you want to create a Spring WebMVC application then in a traditional setup, you would have included all required dependencies yourself. It leaves the chances of version conflict which ultimately result in more runtime exceptions.
With Spring boot, to create MVC application all you need to import is spring-boot-starter-web dependency.
<!-- Parent pom is mandatory to control versions of child dependencies --><parent><groupId>org.springframework.boot</groupId><artifactId>spring-boot-starter-parent</artifactId><version>2.1.6.RELEASE</version><relativePath/></parent><!-- Spring web brings all required dependencies to build web application. --><dependency><groupId>org.springframework.boot</groupId><artifactId>spring-boot-starter-web</artifactId></dependency>
Above spring-boot-starter-web dependency, internally imports all given dependencies and add to your project. Notice how some dependencies are direct, and some dependencies further refer to other starter templates which transitively downloads more dependencies.
Also, notice that you do not need to provide version information into child dependencies. All versions are resolved in relation to version of parent starter (in our example it’s 2.0.4.RELEASE).
Autoconfiguration is enabled with @EnableAutoConfiguration annotation. Spring boot auto configuration scans the classpath, finds the libraries in the classpath and then attempt to guess the best configuration for them, and finally configure all such beans.
Auto-configuration tries to be as intelligent as possible and will back-away as you define more of your own configuration.
Auto-configuration is always applied after user-defined beans have been registered.
Spring boot auto-configuration logic is implemented in spring-boot-autoconfigure.jar. Yoy can verify the list of packages here.
Spring boot autoconfiguration packages
For example, look at auto-configuration for Spring AOP. It does the followings-
Scan classpath to see if EnableAspectJAutoProxy, Aspect, Advice and AnnotatedElement classes are present.
If classes are not present, no autoconfiguration will be made for Spring AOP.
If classes are found then AOP is configured with Java config annotation @EnableAspectJAutoProxy.
It checks for property spring.aop which value can be true or false.
Based on the value of property, proxyTargetClass attribute is set.
Spring boot applications always include tomcat as embedded server dependency. It means you can run the Spring boot applications from the command prompt without needling complex server infrastructure.
You can exclude tomcat and include any other embedded server if you want. Or you can make exclude server environment altogether. It’s all configuration based.
For example, below configuration exclude tomcat and include jetty as embedded server.
To run the application, we need to use @SpringBootApplication annotation. Behind the scenes, that’s equivalent to @Configuration, @EnableAutoConfiguration, and @ComponentScan together.
It enables the scanning of config classes, files and load them into spring context. In below example, execution start with main() method. It start loading all the config files, configure them and bootstrap the application based on application properties in application.properties file in /resources folder.
### Server port #########server.port=8080### Context root ########server.contextPath=/home
To execute the application, you can run the main() method from IDE such eclipse, or you can build the jar file and execute from command prompt.
$ java -jar spring-boot-demo.jar
5. Advantages of Spring boot
Spring boot helps in resolving dependency conflict. It identifies required dependencies and import them for you.
It has information of compatible version for all dependencies. It minimizes the runtime classloader issues.
It’s “opinionated defaults configuration” approach helps you in configuring most important pieces behind the scene. Override them only when you need. Otherwise everything just works, perfectly. It helps in avoiding boilerplate code, annotations and XML configurations.
It provides embedded HTTP server Tomcat so that you can develop and test quickly.
It has excellent integration with IDEs like eclipse and intelliJ idea.
In this article, you’ll learn how to upload and download files in a Restful spring boot web service. The files will be stored in MySQL database.
This article is a continuation of an earlier articlewhere I’ve shown how to upload files and store them in the local filesystem.
We’ll reuse most of the code and concepts described in the last article. So I highly recommend you to go through that before reading this one.
Following is the directory structure of the complete application for your reference –
JPA and MySQL dependencies
Since we’ll be storing files in MySQL database, we’ll need JPA and MySQL dependencies along with Web dependency. So make sure that your pom file contains the following dependencies –
If you’re building the project from scratch, you can generate the project skeleton from Spring Initialzr website
Configuring the Database and Multipart File properties
Next, we need to configure the MySQL database url, username, and password. You can configure that in the src/main/resources/application.properties file –
## Spring DATASOURCE (DataSourceAutoConfiguration & DataSourceProperties)
spring.datasource.url= jdbc:mysql://localhost:3306/file_demo?useSSL=false&serverTimezone=UTC&useLegacyDatetimeCode=false
spring.datasource.username= root
spring.datasource.password= pass
## Hibernate Properties
# The SQL dialect makes Hibernate generate better SQL for the chosen database
spring.jpa.properties.hibernate.dialect = org.hibernate.dialect.MySQL5InnoDBDialect
spring.jpa.hibernate.ddl-auto = update
## Hibernate Logging
logging.level.org.hibernate.SQL= DEBUG
## MULTIPART (MultipartProperties)
# Enable multipart uploads
spring.servlet.multipart.enabled=true
# Threshold after which files are written to disk.
spring.servlet.multipart.file-size-threshold=2KB
# Max file size.
spring.servlet.multipart.max-file-size=200MB
# Max Request Size
spring.servlet.multipart.max-request-size=215MB
The above properties file also has Multipart file properties. You can make changes to these properties as per your requirements.
DBFile model
Let’s create a DBFile entity to model the file attributes that will be stored in the database –
I’ll be reusing the complete front-end code that I demonstrated in the first part of this article. You should check out the first article to learn more about the front-end code.
Once you have the front-end code in place, type the following command to run the application –
One of the most common, and efficient, models deployed by enterprises is the Layered Architecture, also called the n-tiered pattern. It packs similar components together in a horizontal manner and is self-independent. What does that mean?
It implies that the layers of the model are interconnected to each other but not interdependent. Similar components stay at the same level allowing the layers to be separated inadvertently based on the nature of the code. It is this isolation, that lends the software layers an independent nature.
Consider an instance, wherein you’d want to switch from an Oracle database to an SQL. This shift may cause you to upend the database layer but will not have a domino effect on any other layer.
Evidently, it serves a challenge for an enterprise software architect to create layers that separate from each other. Nevertheless, since the roles of each layer are clearly distinct, it accredits this software development architecture the following qualities:
It is easily maintainable as enterprise software developers with limited, or should we say pertinent, knowledge can be assigned to operate on a single layer.
You can test changes in layers separately from each other.
Upgraded versions of the software can be implemented effortlessly.
The flow of code is top-down, meaning it enters the presentation layer first and trickles down to the bottom-most layer that is the database layer. Each layer has a designated task based on the nature of the components it preserves. These could be checking the consistency of values within the code or reformatting the code altogether.
Refactoring – a key – way to lower frontend maintenance cost is a software development process by which developers change the internal shape and size of the code. They do it without affecting its external attributes can also be carried out in an n-tiered model.
This software development architecture can be customized to add layers to the presentation, business, persistence, and database levels. Such a model is called a Hybrid Layered architecture.
Benefits
Amongst the various types of software architecture, the layered variant suits enterprises that don’t want to go overboard with experimenting and want to stick to the traditional software architecture design patterns.
Testing components become relatively easier as inter-dependencies are negligible in this format of software development engineering.
Considering many software frameworks were built with the backdrop of an n-tiered structure, applications built with them, as a result, happen to be in the layered format as well.
Potential Drawbacks
Larger applications tend to be resource-intensive if based on this format, therefore for such projects, it is advised to overlook the layered pattern.
Although the layers are independent, yet the entire version of the software is installed as a single unit. Therefore, even if you update a single layer, you would have to re-install the entire apparatus all over again.
Such systems are not scalable due to the coupling between the layers.
Ideal For
The Layered architecture pattern suits the niche of LOB i.e. Line of Business Applications. These are applications that are essential to the functioning of the business itself. For instance, the accounts department of an organization needs software such as QuickBooks, Xero, Sage or Wave Accounting for keeping financial data.
Similarly, the marketing team would demand a customer relationship management software slash tool to help them cope with the volume of interactions. In short, applications that do more than just CRUD (create, read, update, and delete) operations are suited to the layered architecture pattern.
External configurations allow you to work with the same code in different environments. They also provide you the flexibility to tune your application from a single place.
In this article, you’ll learn how to define and use external configurations in Spring Boot with a very simple annotation based API called @ConfigurationProperties.
@ConfigurationProperties bind external configurations to a strongly typed bean in your application code. You can inject and use this bean throughout your application code just like any other spring bean.
Let the exploring begin!
Creating the Application
Let’s create a sample application to learn how to define and use external configurations in spring boot.
We’ll use Spring Boot CLI to initialize the project. Fire up your terminal and type the following command to generate the project –
spring init --dependencies=web --name=config-properties-demo --package-name=com.example.demo config-properties-demo
Alternatively, You may also bootstrap the application from Spring Initializr website by following the instructions below –
Enter config-properties-demo in the Artifact field.
Add Web in the dependencies section.
Click Generate Project to download the project.
Following is the directory structure of the complete application for your reference-
Defining Properties
Spring Boot application loads configuration properties from application.properties file located in the classpath by default.
Open src/main/resources/application.properties file and add the following properties to it
## Top level app properties
app.name=ConfigurationPropertiesDemoApp
app.description=${app.name} is a spring boot app that demonstrates how to use external configuration properties
app.upload-dir=/uploads
app.connect-timeout=500ms
app.read-timeout=10s
## Nested Object Properties (security)
app.security.username=user
app.security.password=123456
app.security.roles=USER,ADMIN,PARTNER # List Property
app.security.enabled=true
## Map Properties (permissions)
app.security.permissions.CAN_VIEW_POSTS=true
app.security.permissions.CAN_EDIT_POSTS=true
app.security.permissions.CAN_DELETE_POSTS=false
app.security.permissions.CAN_VIEW_USERS=true
app.security.permissions.CAN_EDIT_USERS=true
app.security.permissions.CAN_DELETE_USERS=false
Binding external properties to a POJO class using @ConfigurationProperties
Let’s now see how we can bind the properties defined in the application.properties file to a POJO class using @ConfigurationProperties.
The @ConfigurationProperties annotation takes a prefix parameter, and binds all the properties with the specified prefix to the POJO class –
package com.example.demo.config;
import org.springframework.boot.context.properties.ConfigurationProperties;
import org.springframework.boot.convert.DurationUnit;
import java.time.Duration;
import java.time.temporal.ChronoUnit;
import java.util.ArrayList;
import java.util.HashMap;
import java.util.List;
import java.util.Map;
@ConfigurationProperties(prefix = "app")
public class AppProperties {
private String name;
private String description;
private String uploadDir;
private Duration connectTimeout = Duration.ofMillis(1000);
@DurationUnit(ChronoUnit.SECONDS)
private Duration readTimeout = Duration.ofSeconds(30);
private final Security security = new Security();
// Getters and Setters (Omitted for brevity)
public static class Security {
private String username;
private String password;
private List<String> roles = new ArrayList<>();
private boolean enabled;
private Map<String, String> permissions = new HashMap<>();
// Getters and Setters (Omitted for brevity)
}
}
Let’s understand a few details about the binding:
Type-safe binding (List and Map)Notice how the comma separated roles in the properties file are bound to a List of roles and the permissions are bound to a Map.
Duration SupportAlso, note how the duration properties are safely bound to the Duration types. This is so awesome. Spring Boot allows you to specify durations with the following units in the application.properties files –
ns for nanoseconds
us for microseconds
ms for milliseconds
s for seconds
m for minutes
h for hours
d for days
The default unit is milliseconds. So if you don’t specify any unit in the properties file, It will be considered as milliseconds.Note that, you can also override the unit using @DurationUnit annotation as we have done in the above POJO class.
Naming ConventionAll the kebab case property names (ex: upload-dir) are bound to the corresponding camel case fields (ex: uploadDir) in the POJO class.Note that the properties can also be specified in camel case. But using kebab case is recommended.
Enabling Configuration Properties
You need to explicitly register the properties classes using the @EnableConfigurationProperties annotation as shown in the following example.
package com.example.demo;
import com.example.demo.config.AppProperties;
import org.springframework.boot.SpringApplication;
import org.springframework.boot.autoconfigure.SpringBootApplication;
import org.springframework.boot.context.properties.EnableConfigurationProperties;
@SpringBootApplication
@EnableConfigurationProperties(AppProperties.class)
public class ConfigPropertiesDemoApplication {
public static void main(String[] args) {
SpringApplication.run(ConfigPropertiesDemoApplication.class, args);
}
}
Alternatively, you could also add a @Component annotation to the AppProperties class and the binding would still work.
Injecting Configuration Properties in your Spring Beans
The @ConfigurationProperties classes are regular spring beans, and you can inject them in the same way as any other bean.
In the following example, I’ve written a sample API that retrieves app details from configuration properties and returns them to the client –
package com.example.demo.controller;
import com.example.demo.config.AppProperties;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.web.bind.annotation.GetMapping;
import org.springframework.web.bind.annotation.RestController;
import java.util.HashMap;
import java.util.Map;
@RestController
public class IndexController {
// Injecting ConfigurationProperties in your Beans
@Autowired
private AppProperties appProperties;
@GetMapping("/")
public Map<String, String> getAppDetails() {
Map<String, String> appDetails = new HashMap<>();
appDetails.put("name", appProperties.getName());
appDetails.put("description", appProperties.getDescription());
return appDetails;
}
}
@ConfigurationProperties Validation
You can validate configuration properties using javax.validation constraints like @NotNull, @NotEmpty etc.
To enable validation, you just need to add Spring’s @Validated annotation to the @ConfigurationProperties class –
package com.example.demo.config;
import org.springframework.boot.context.properties.ConfigurationProperties;
import org.springframework.boot.convert.DurationUnit;
import org.springframework.validation.annotation.Validated;
import javax.validation.Valid;
import javax.validation.constraints.NotEmpty;
import javax.validation.constraints.NotNull;
import java.time.Duration;
import java.time.temporal.ChronoUnit;
import java.util.ArrayList;
import java.util.HashMap;
import java.util.List;
import java.util.Map;
@ConfigurationProperties(prefix = "app")
@Validated
public class AppProperties {
@NotNull
private String name;
private String description;
private String uploadDir;
private Duration connectTimeout = Duration.ofMillis(1000);
@DurationUnit(ChronoUnit.SECONDS)
private Duration readTimeout = Duration.ofSeconds(30);
@Valid
private final Security security = new Security();
// Getters and Setters (Omitted for brevity)
public static class Security {
private String username;
private String password;
@NotEmpty
private List<String> roles = new ArrayList<>();
private boolean enabled;
private Map<String, String> permissions = new HashMap<>();
// Getters and Setters (Omitted for brevity)
}
}
Now, The Spring Boot application will throw a validation exception on startup, if the name property is null or the security.roles property is empty
Environment based (Profile specific) Configuration Properties
In addition to the standard application.properties file, Spring Boot also allows you to define profile-specific properties with the following naming convention –
application-{profile}.properties
The profile specific property files are loaded from the same location as the application.properties file, with profile-specific properties always overriding the default ones.
To illustrate this, Let’s define some profile-specific property files, and override some of the default properties –
application-dev.properties## Override properties for dev environment app.name=ConfigurationPropertiesDemoApp-DEVELOPMENT app.security.username=project-dev
application-staging.properties## Override properties for staging environment app.name=ConfigurationPropertiesDemoApp-STAGING app.security.username=project-staging app.security.password=C@ll1C0d3r
application-prod.properties## Override properties for prod environment app.name=ConfigurationPropertiesDemoApp-PRODUCTION app.security.username=project-prod app.security.password=C@ll1C0d3r
Now, we need to activate a spring profile so that the corresponding properties file is loaded by spring boot.
Activating a Spring Profile
You can set active profiles in many ways –
Using application propertiesThe default application.properties file is always loaded by Spring Boot. You can set active profiles in the application.properties file by adding the following property –spring.profiles.active=staging After adding the above property, Spring Boot will load application-staging.properties file as well, and override properties with the new values specified there.
Using command line argumentYou can set active profiles on startup by providing the spring.profiles.active command line argument like this –# Packaging the app mvn clean package -Dspring.profiles.active=staging # Running the packaged jar with `spring.profiles.active` argument java -jar -Dspring.profiles.active=staging target/config-properties-demo-0.0.1-SNAPSHOT.jar Moreover, Here is how you can set active profiles while running the application with spring-boot:run command –mvn spring-boot:run -Dspring.profiles.active=dev
Using environment variableLastly, you can also set active profiles using the SPRING_PROFILES_ACTIVE environment variable-export SPRING_PROFILES_ACTIVE=prod
Running the application
Let’s run the application and access the sample Rest API to see configuration properties in action –
mvn spring-boot:run
$ curl http://localhost:8080
{"name":"ConfigurationPropertiesDemoApp","description":"ConfigurationPropertiesDemoApp is a spring boot app that demonstrates how to use external configuration properties"}
Running the application with active profiles set to prod
mvn spring-boot:run -Dspring.profiles.active=prod
$ curl http://localhost:8080
{"name":"ConfigurationPropertiesDemoApp-PRODUCTION","description":"ConfigurationPropertiesDemoApp-PRODUCTION is a spring boot app that demonstrates how to use external configuration properties"}
Conclusion
@ConfigurationProperties are a really nice way to bind external configurations in a type-safe manner. I use this feature in almost all my projects. Moreover, Spring Boot’s auto-configurations also rely on configuration properties.
Let me know what do you think about this feature in the comment section below.
Spring Boot is an opinionated, convention-over-configuration focused addition to the Spring platform – highly useful to get started with minimum effort and create stand-alone, production-grade applications.
This tutorial is a starting point for Boot – a way to get started in a simple manner, with a basic web application.
We’ll go over some core configuration, a front-end, quick data manipulation, and exception handling.
2. Setup
First, let’s use Spring Initializr to generate the base for our project.
Next, we’ll configure a simple main class for our application:
@SpringBootApplication
public class Application {
public static void main(String[] args) {
SpringApplication.run(Application.class, args);
}
}
Notice how we’re using @SpringBootApplication as our primary application configuration class; behind the scenes, that’s equivalent to @Configuration, @EnableAutoConfiguration, and @ComponentScan together.
Finally, we’ll define a simple application.properties file – which for now only has one property:
server.port=8081
server.port changes the server port from the default 8080 to 8081; there are of course many more Spring Boot properties available.
4. Simple MVC View
Let’s now add a simple front end using Thymeleaf.
First, we need to add the spring-boot-starter-thymeleaf dependency to our pom.xml:
By now, you’re hopefully noticing a pattern – most Spring libraries are easily imported into our project with the use of simple Boot starters.
Once the spring-boot-starter-security dependency on the classpath of the application – all endpoints are secured by default, using either httpBasic or formLogin based on Spring Security’s content-negotiation strategy.
That’s why, if we have the starter on the classpath, we should usually define our own custom Security configuration by extending the WebSecurityConfigurerAdapter class:
In our example, we’re allowing unrestricted access to all endpoints.
Of course, Spring Security is an extensive topic and one not easily covered in a couple of lines of configuration – so I definitely encourage you to go deeper into the topic.
6. Simple Persistence
Let’s start by defining our data model – a simple Book entity:
@Entity
public class Book {
@Id
@GeneratedValue(strategy = GenerationType.AUTO)
private long id;
@Column(nullable = false, unique = true)
private String title;
@Column(nullable = false)
private String author;
}
And its repository, making good use of Spring Data here:
Finally, we need to of course configure our new persistence layer:
@EnableJpaRepositories("com.baeldung.persistence.repo")
@EntityScan("com.baeldung.persistence.model")
@SpringBootApplication
public class Application {
...
}
Note that we’re using:
@EnableJpaRepositories to scan the specified package for repositories
@EntityScan to pick up our JPA entities
To keep things simple, we’re using an H2 in-memory database here – so that we don’t have any external dependencies when we run the project.
Once we include H2 dependency, Spring Boot auto-detects it and sets up our persistence with no need for extra configuration, other than the data source properties:
Of course, like security, persistence is a broader topic than this basic set here, and one you should certainly explore further.
7. Web and the Controller
Next, let’s have a look at a web tier – and we’ll start that by setting up a simple controller – the BookController.
We’ll implement basic CRUD operations exposing Book resources with some simple validation:
@RestController
@RequestMapping("/api/books")
public class BookController {
@Autowired
private BookRepository bookRepository;
@GetMapping
public Iterable findAll() {
return bookRepository.findAll();
}
@GetMapping("/title/{bookTitle}")
public List findByTitle(@PathVariable String bookTitle) {
return bookRepository.findByTitle(bookTitle);
}
@GetMapping("/{id}")
public Book findOne(@PathVariable Long id) {
return bookRepository.findById(id)
.orElseThrow(BookNotFoundException::new);
}
@PostMapping
@ResponseStatus(HttpStatus.CREATED)
public Book create(@RequestBody Book book) {
return bookRepository.save(book);
}
@DeleteMapping("/{id}")
public void delete(@PathVariable Long id) {
bookRepository.findById(id)
.orElseThrow(BookNotFoundException::new);
bookRepository.deleteById(id);
}
@PutMapping("/{id}")
public Book updateBook(@RequestBody Book book, @PathVariable Long id) {
if (book.getId() != id) {
throw new BookIdMismatchException();
}
bookRepository.findById(id)
.orElseThrow(BookNotFoundException::new);
return bookRepository.save(book);
}
}
Given this aspect of the application is an API, we made use of the @RestController annotation here – which equivalent to a @Controller along with @ResponseBody – so that each method marshalls the returned resource right to the HTTP response.
Just one note worth pointing out – we’re exposing our Book entity as our external resource here. That’s fine for our simple application here, but in a real-world application, you will likely want to separate these two concepts.
8. Error Handling
Now that the core application is ready to go, let’s focus on a simple centralized error handling mechanism using @ControllerAdvice:
@ControllerAdvice
public class RestExceptionHandler extends ResponseEntityExceptionHandler {
@ExceptionHandler({ BookNotFoundException.class })
protected ResponseEntity<Object> handleNotFound(
Exception ex, WebRequest request) {
return handleExceptionInternal(ex, "Book not found",
new HttpHeaders(), HttpStatus.NOT_FOUND, request);
}
@ExceptionHandler({ BookIdMismatchException.class,
ConstraintViolationException.class,
DataIntegrityViolationException.class })
public ResponseEntity<Object> handleBadRequest(
Exception ex, WebRequest request) {
return handleExceptionInternal(ex, ex.getLocalizedMessage(),
new HttpHeaders(), HttpStatus.BAD_REQUEST, request);
}
}
Beyond the standard exceptions we’re handling here, we’re also using a custom exception:
BookNotFoundException:
public class BookNotFoundException extends RuntimeException {
public BookNotFoundException(String message, Throwable cause) {
super(message, cause);
}
// ...
}
This should give you an idea of what’s possible with this global exception handling mechanism. If you’d like to see a full implementation, have a look at the in-depth tutorial.
Note that Spring Boot also provides an /error mapping by default. We can customize its view by creating a simple error.html:
Spring Boot provides sensible defaults for many configuration properties. But we sometimes need to customize these with our case-specific values.
And a common use case is changing the default port for the embedded server.
In this quick tutorial, we’ll cover several ways to achieve this.
2. Using Property Files
The fastest and easiest way to customize Spring Boot is by overriding the values of the default properties.
For the server port, the property we want to change is server.port.
By default, the embedded server starts on port 8080.
So, let’s see how to provide a different value in an application.properties file:
server.port=8081
Now the server will start on port 8081.
And we can do the same if we’re using an application.yml file:
server:
port : 8081
Both files are loaded automatically by Spring Boot if placed in the src/main/resources directory of a Maven application.
2.1. Environment-Specific Ports
If we have an application deployed in different environments, we may want it to run on different ports on each system.
We can easily achieve this by combining the property files approach with Spring profiles. Specifically, we can create a property file for each environment.
For example, we’ll have an application-dev.properties file with this content:
server.port=8081
Then we’ll add another application-qa.properties file with a different port:
server.port=8082
Now, the property files configuration should be sufficient for most cases. However, there are other options for this goal, so let’s explore them as well.
3. Programmatic Configuration
We can configure the port programmatically either by setting the specific property when starting the application or by customizing the embedded server configuration.
First, let’s see how to set the property in the main @SpringBootApplication class:
@SpringBootApplication
public class CustomApplication {
public static void main(String[] args) {
SpringApplication app = new SpringApplication(CustomApplication.class);
app.setDefaultProperties(Collections
.singletonMap("server.port", "8083"));
app.run(args);
}
}
Next, to customize the server configuration, we have to implement the WebServerFactoryCustomizer interface:
@Component
public class ServerPortCustomizer
implements WebServerFactoryCustomizer<ConfigurableWebServerFactory> {
@Override
public void customize(ConfigurableWebServerFactory factory) {
factory.setPort(8086);
}
}
Note that this applies to the Spring Boot 2.x version.
For Spring Boot 1.x, we can similarly implement the EmbeddedServletContainerCustomizer interface.
4. Using Command-Line Arguments
When packaging and running our application as a jar, we can set the server.port argument with the java command:
java -jar spring-5.jar --server.port=8083
or by using the equivalent syntax:
java -jar -Dserver.port=8083 spring-5.jar
5. Order of Evaluation
As a final note, let’s look at the order in which these approaches are evaluated by Spring Boot.
Basically, the configurations priority is
embedded server configuration
command-line arguments
property files
main @SpringBootApplication configuration
6. Conclusion
In this article, we saw how to configure the server port in a Spring Boot application.
Discover the deliberate design decisions that have made Kafka the performance powerhouse it is today.
The last few years have brought about immense changes in the software architecture landscape. The notion of a single monolithic application or even several coarse-grained services sharing a common data store has been all but erased from the hearts and minds of software practitioners world-wide. Autonomous microservices, event-driven architecture, and CQRS are the dominant tools in the construction of contemporary business-centric applications. To top it off, the proliferation of device connectivity — IoT, mobile, wearables — is creating an upward pressure on the number of events a system must handle in near-real-time.
Let’s start by acknowledging that the term ‘fast’ is multi-faceted, complex, and highly ambiguous. Latency, throughput, jitter, are metrics that shape and influence one’s interpretation of the term. It is also inherently contextual: the industry and application domains in themselves set the norms and expectations around performance. Whether or not something is fast depends largely on one’s frame of reference.
Apache Kafka is optimized for throughput at the expense of latency and jitter, while preserving other desirable qualities, such as durability, strict record order, and at-least-once delivery semantics. When someone says ‘Kafka is fast’, and assuming they are at least mildly competent, you can assume they are referring to Kafka’s ability to safely accumulate and distribute a very high number of records in a short amount of time.
Historically, Kafka was born out of LinkedIn’s need to move a very large number of messages efficiently, amounting to multiple terabytes of data on an hourly basis. The individual message propagation delay was deemed of secondary importance, as was the variability of that time. After all, LinkedIn is not a financial institution that engages in high-frequency trading, nor is it an industrial control system that operates within deterministic deadlines. Kafka can be used to implement near-real-time (otherwise known as soft real-time) systems.
Note: For those unfamiliar with the term, ‘real-time’ does not mean ‘fast’, it means ‘predictable’. Specifically, real-time implies a hard upper bound, otherwise known as a deadline, on the time taken to complete an action. If the system as a whole is unable to meet this deadline each and every time, it cannot be classed as real-time. Systems that are able to perform within a probabilistic tolerance, are labelled as ‘near-real-time’. In terms of sheer throughput, real-time systems are often slower than their near-real-time or non-real-time counterparts.
There are two significant areas that Kafka draws upon for its speed, and they need to be discussed separately. The first relates to the low-level efficiency of the client and broker implementations. The second derives from the opportunistic parallelism of stream processing.
Broker performance
Log-structured persistence
Kafka utilizes a segmented, append-only log, largely limiting itself to sequential I/O for both reads and writes, which is fast across a wide variety of storage media. There is a wide misconception that disks are slow; however, the performance of storage media (particularly rotating media) is greatly dependent on access patterns. The performance of random I/O on a typical 7,200 RPM SATA disk is between three and four orders of magnitude slower when compared to sequential I/O. Furthermore, a modern operating system provides read-ahead and write-behind techniques that prefetch data in large block multiples and group smaller logical writes into large physical writes. Because of this, the difference between sequential I/O and random I/O is still evident in flash and other forms of solid-state non-volatile media, although it is far less dramatic compared to rotating media.
Record batching
Sequential I/O is blazingly fast on most media types, comparable to the peak performance of network I/O. In practice, this means that a well-designed log-structured persistence layer will keep up with the network traffic. In fact, quite often the bottleneck with Kafka isn’t the disk, but the network. So in addition to the low-level batching provided by the OS, Kafka clients and brokers will accumulate multiple records in a batch — for both reading and writing — before sending them over the network. Batching of records amortizes the overhead of the network round-trip, using larger packets and improving bandwidth efficiency.
Batch compression
The impact of batching is particularly obvious when compression is enabled, as compression becomes generally more effective as the data size increases. Especially when using text-based formats such as JSON, the effects of compression can be quite pronounced, with compression ratios typically ranging from 5x to 7x. Furthermore, record batching is largely done as a client-side operation, which transfers the load onto the client and has a positive effect not only on the network bandwidth but also on the brokers’ disk I/O utilization.
Cheap consumers
Unlike traditional MQ-style brokers which remove messages at point of consumption (incurring the penalty of random I/O), Kafka doesn’t remove messages after they are consumed — instead, it independently tracks offsets at each consumer group level. The progression of offsets themselves is published on an internal Kafka topic __consumer_offsets. Again, being an append-only operation, this is fast. The contents of this topic are further reduced in the background (using Kafka’s compaction feature) to only retain the last known offsets for any given consumer group.
Compare this model with more traditional message brokers which typically offer several contrasting message distribution topologies. On one hand is the message queue — a durable transport for point-to-point messaging, with no point-to-multipoint ability. On the other hand, a pub-sub topic allows for point-to-multipoint messaging but does so at the expense of durability. Implementing a durable point-to-multipoint messaging model in a traditional MQ requires maintaining a dedicated message queue for each stateful consumer. This creates both read and write amplification. On one hand, the publisher is forced to write to multiple queues. Alternatively, a fan-out relay may consume records from one queue and write to several others, but this only defers the point of amplification. On the other hand, several consumers are generating load on the broker — being a mixture of read and write I/O, both sequential and random.
Consumers in Kafka are ‘cheap’, insofar as they don’t mutate the log files (only the producer or internal Kafka processes are permitted to do that). This means that a large number of consumers may concurrently read from the same topic without overwhelming the cluster. There is still some cost in adding a consumer, but it is mostly sequential reads with a low rate of sequential writes. So it’s fairly normal to see a single topic being shared across a diverse consumer ecosystem.
Unflushed buffered writes
Another fundamental reason for Kafka’s performance, and one that is worth exploring further: Kafka doesn’t actually call fsync when writing to the disk before acknowledging the write; the only requirement for an ACK is that the record has been written to the I/O buffer. This is a little known fact, but a crucial one: in fact, this is what actually makes Kafka perform as if it were an in-memory queue — because for all intents and purposes Kafka is a disk-backed in-memory queue (limited by the size of the buffer/pagecache).
On the flip side, this form of writing is unsafe, as the failure of a replica can lead to a data loss even though the record has seemingly been acknowledged. In other words, unlike say a relational database, acknowledging a write alone does not imply durability. What makes Kafka durable is running several in-sync replicas; even if one were to fail, the others (assuming there is more than one) will remain operational — providing that the failure is uncorrelated (i.e. multiple replicas failing simultaneously due of a common upstream failure). So the combination of a non-blocking approach to I/O with no fsync, and redundant in-sync replicas give Kafka the combination of high throughput, durability, and availability.
Client-side optimisations
Most databases, queues, and other forms of persistent middleware are designed around the notion of an all-mighty server (or a cluster of servers), and fairly thin clients that communicate with the server(s) over a well-known wire protocol. Client implementations are generally considered to be significantly simpler than the server. As a result, the server will absorb the bulk of the load — the clients merely act as interfaces between the application code and the server.
Kafka takes a different approach to client design. A significant amount of work is performed on the client before records get to the server. This includes the staging of records in an accumulator, hashing the record keys to arrive at the correct partition index, checksumming the records and the compression of the record batch. The client is aware of the cluster metadata and periodically refreshes this metadata to keep abreast of any changes to the broker topology. This lets the client make low-level forwarding decisions; rather than sending a record blindly to the cluster and relying on the latter to forward it to the appropriate broker node, a producer client will forward writes directly to partition masters. Similarly, consumer clients are able to make intelligent decisions when sourcing records, potentially using replicas that geographically closer to the client when issuing read queries. (This feature is a more recent addition to Kafka, available as of version 2.4.0.)
Zero-copy
One of the typical sources of inefficiencies is copying byte data between buffers. Kafka uses a binary message format that is shared by the producer, the broker, and the consumer parties so that data chunks can flow end-to-end without modification, even if it’s compressed. While eliminating structural differences between communicating parties is an important step, it doesn’t in itself avoid the copying of data.
Kafka solves this problem on Linux and UNIX systems by using Java’s NIO framework, specifically, the transferTo() method of a java.nio.channels.FileChannel. This method permits the transfer of bytes from a source channel to a sink channel without involving the application as a transfer intermediary. To appreciate the difference that NIO makes, consider the traditional approach where a source channel is read into a byte buffer, then written to a sink channel as two separate operations:
Diagrammatically, this can be represented using the following.
Although this looks simple enough, internally, the copy operation requires four context switches between user mode and kernel mode, and the data is copied four times before the operation is complete. The diagram below outlines the context switches at each step.
Looking at this in more detail —
The initial read() causes a context switch from user mode to kernel mode. The file is read, and its contents are copied to a buffer in the kernel address space by the DMA (Direct Memory Access) engine. This is not the same buffer that was used in the code snippet.
Prior to returning from read(), the kernel buffer is copied into the user-space buffer. At this point, our application can read the contents of the file.
The subsequent send() will switch back into kernel mode, copying the user-space buffer into the kernel address space — this time into a different buffer associated with the destination socket. Behind the scenes, the DMA engine takes over, asynchronously copying the data from the kernel buffer to the protocol stack. The send() method does not wait for this prior to returning.
The send() call returns, switching back to the user-space context.
In spite of its mode-switching inefficiencies and additional copying, the intermediate kernel buffer can actually improve performance in many cases. It can act as a read-ahead cache, asynchronously prefetching blocks, thereby front-running requests from the application. However, when the amount of requested data is significantly larger than the kernel buffer size, the kernel buffer becomes a performance bottleneck. Rather than copying the data directly, it forces the system to oscillate between user and kernel modes until all the data is transferred.
By contrast, the zero-copy approach is handled in a single operation. The snippet from the earlier example can be rewritten as a one-liner:
fileDesc.transferTo(offset, len, socket);
The zero-copy approach is illustrated below.
Under this model, the number of context switches is reduced to one. Specifically, the transferTo() method instructs the block device to read data into a read buffer by the DMA engine. This buffer is then copied another kernel buffer for staging to the socket. Finally, the socket buffer is copied to the NIC buffer by DMA.
As a result, we have reduced the number of copies from four to three, and only one of those copies involves the CPU. We have also reduced the number of context switches from four to two.
This is a massive improvement, but it’s not query zero-copy yet. The latter can be achieved as a further optimization when running Linux kernels 2.4 and later, and on network interface cards that support the gather operation. This is illustrated below.
Calling the transferTo() method causes the device to read data into a kernel read buffer by the DMA engine, as per the previous example. However, with the gather operation, there is no copying between the read buffer and the socket buffer. Instead, the NIC is given a pointer to the read buffer, along with the offset and the length, which is vacuumed up by DMA. At no point is the CPU involved in copying buffers.
Comparisons of traditional and zero-copy on file sizes ranging from a few megabytes to a gigabyte show performance gains by a factor of two to three in favor of zero-copy. But what’s more impressive, is that Kafka achieves this using a plain JVM with no native libraries or JNI code.
Avoiding the GC
The heavy use of channels, native buffers, and the page cache has one additional benefit — reducing the load on the garbage collector (GC). For example, running Kafka on a machine with 32 GB of RAM will result in 28–30 GB usable for the page cache, completely outside of the GC’s scope. The difference in throughput is minimal — in the region of several percentage points — as the throughput of a correctly-tuned GC can be quite high, especially when dealing with short-lived objects. The real gains are in the reduction of jitter; by avoiding the GC, the brokers are less likely to experience a pause that may impact the client, extending the end-to-end propagation delay of records.
To be fair, the avoidance of GC is less of a problem now, compared to what it used to be when Kafka was conceived. Modern GCs like Shenandoah and ZGC scale to huge, multi-terabyte heaps, and have tunable worst-case pause times, down to single-digit milliseconds. It is not uncommon these days to see JVM-based applications using large heap-based caches outperform off-heap designs.
Stream parallelism
The efficiency of log-structured I/O is one crucial aspect of performance, mostly affecting writes; Kafka’s treatment of parallelism in the topic structure and the consumer ecosystem is fundamental to its read performance. The combination produces an overall very high end-to-end messaging throughput. Concurrency is ingrained into its partitioning scheme and the operation of consumer groups, which is effectively a load-balancing mechanism within Kafka — distributing partition assignments approximately evenly among the individual consumer instances within the group. Compare this to a more traditional MQ: in an equivalent RabbitMQ setup, multiple concurrent consumers may read from a queue in a round-robin fashion, but in doing so they forfeit the notion of message ordering.
The partitioning mechanism also allows for the horizontal scalability of Kafka brokers. Every partition has a dedicated leader; any nontrivial topic (with multiple partitions) can, therefore, utilize the entire cluster of broker for writes. This is yet another point of distinction between Kafka and a message queue; where the latter utilizes clustering for availability, Kafka will genuinely balance the load across the brokers for availability, durability, and throughput.
The producer specifies the partition when publishing a record, assuming that you are publishing to a topic with multiple partitions. (One may have a single-partition topic, in which case this is a non-issue.) This may be accomplished either directly — by specifying a partition index, or indirectly — by way of a record key, which deterministically hashes to a consistent (i.e. same every time) partition index. Records sharing the same hash are guaranteed to occupy the same partition. Assuming a topic with multiple partitions, records with a different key will likely end up in different partitions. However, due to hash collisions, records with different hashes may also end up in the same partition. Such is the nature of hashing. If you understand how a hash table works, this is no different.
The actual processing of records is done by consumers, operating within an (optional) consumer group. Kafka guarantees that a partition may only be assigned to at most one consumer within its consumer group. (We say ‘at most’ to cover the case when all consumers are offline.) When the first consumer in a group subscribes to the topic, it will receive all partitions on that topic. When a second consumer subsequently joins, it will get approximately half of the partitions, relieving the first consumer of half of its prior load. This enables you to process an event stream in parallel, adding consumers as necessary (ideally, using an auto-scaling mechanism), providing that you have adequately partitioned your event stream.
Control of record throughput accomplished in two ways:
The topic partitioning scheme. Topics should be partitioned to maximize the number of independent event sub-streams. In other words, record order should only be preserved where it is absolutely necessary. If any two records are not legitimately related in a causal sense, they shouldn’t be bound to the same partition. This implies the use of different keys, as Kafka will use a record’s key as a hashing source to derive its consistent partition mapping.
The number of consumers in the group. You can increase the number of consumers to match the load of inbound records, up to the number of partitions in the topic. (You can have more consumers if you wish, but the partition count will place an upper bound on the number of active consumers which get at least one partition assignment; the remaining consumers will remain idle.) Note that a consumer could be a process or a thread. Depending on the type of workload that the consumer performs, you may be able to employ multiple individual consumer threads or process records in a thread pool.
If you were wondering whether Kafka is fast, how it achieves its renowned performance characteristics, or if it can scale to your use cases, you should hopefully by now have all the answers you need.
To make things abundantly clear, Kafka is not the fastest (that is, most throughput-capable) messaging middleware — there are other platforms capable of greater throughput — some are software-based and some are implemented in hardware. Nor is it the best throughput-latency compromise — Apache Pulsar is a promising technology that is scalable and achieves a better throughput-latency profile while offering identical ordering and durability guarantees. The rationale for adopting Kafka is that as a complete ecosystem, it remains unmatched overall. It exhibits excellent performance while offering an environment that is abundant and mature, but also involving — in spite of its size, Kafka is still growing at an enviable pace.
The designers and maintainers of Kafka have done an amazing job at devising a solution that is performance-oriented at its core. Few of its design elements feel like an afterthought or a bolt-on. From offloading of work to clients to the log-structured persistence on the broker, batching, compression, zero-copy I/O, and stream-level parallelism — Kafka throws down the gauntlet to just about any other message-oriented middleware, commercial or open-source. And most impressively, it does so without compromising on qualities such as durability, record order, and at-least-once delivery semantics.
Kafka is not the simplest of messaging platforms, and there is a fair bit to learn. One must come to grips with the concepts of a total and partial order, topics, partitions, consumers and consumer groups, before comfortably designing and building high-performance event-driven systems. And while the knowledge curve is substantial, the results will certainly be worth your while. If you are keen on taking the proverbial ‘red pill’, read the Introduction to Event Streaming with Kafka and Kafdrop.