Category: Spring Boot

The Spring Framework is an application framework and inversion of control container for the Java platform. The framework’s core features can be used by any Java application, but there are extensions for building web applications on top of the Java EE platform.

Getting started with Spring Boot
26
Mar
2021

Getting started with Spring Boot

1. Overview

Spring Boot is an opinionated, convention-over-configuration focused addition to the Spring platform – highly useful to get started with minimum effort and create stand-alone, production-grade applications.

This tutorial is a starting point for Boot – a way to get started in a simple manner, with a basic web application.

We’ll go over some core configuration, a front-end, quick data manipulation, and exception handling.

2. Setup

First, let’s use Spring Initializr to generate the base for our project.

The generated project relies on the Boot parent:

<parent>
    <groupId>org.springframework.boot</groupId>
    <artifactId>spring-boot-starter-parent</artifactId>
    <version>2.4.0</version>
    <relativePath />
</parent>

The initial dependencies are going to be quite simple:

<dependency>
    <groupId>org.springframework.boot</groupId>
    <artifactId>spring-boot-starter-web</artifactId>
</dependency>
<dependency>
    <groupId>org.springframework.boot</groupId>
    <artifactId>spring-boot-starter-data-jpa</artifactId>
</dependency>
<dependency>
    <groupId>com.h2database</groupId>
    <artifactId>h2</artifactId>
</dependency>

3. Application Configuration

Next, we’ll configure a simple main class for our application:

@SpringBootApplication
public class Application {
    public static void main(String[] args) {
        SpringApplication.run(Application.class, args);
    }
}

Notice how we’re using @SpringBootApplication as our primary application configuration class; behind the scenes, that’s equivalent to @Configuration@EnableAutoConfiguration, and @ComponentScan together.

Finally, we’ll define a simple application.properties file – which for now only has one property:

server.port=8081

server.port changes the server port from the default 8080 to 8081; there are of course many more Spring Boot properties available.

4. Simple MVC View

Let’s now add a simple front end using Thymeleaf.

First, we need to add the spring-boot-starter-thymeleaf dependency to our pom.xml:

<dependency> 
    <groupId>org.springframework.boot</groupId> 
    <artifactId>spring-boot-starter-thymeleaf</artifactId> 
</dependency>

That enables Thymeleaf by default – no extra configuration is necessary.

We can now configure it in our application.properties:

spring.thymeleaf.cache=false
spring.thymeleaf.enabled=true 
spring.thymeleaf.prefix=classpath:/templates/
spring.thymeleaf.suffix=.html

spring.application.name=Bootstrap Spring Boot

Next, we’ll define a simple controller and a basic home page – with a welcome message:

@Controller
public class SimpleController {
    @Value("${spring.application.name}")
    String appName;

    @GetMapping("/")
    public String homePage(Model model) {
        model.addAttribute("appName", appName);
        return "home";
    }
}

Finally, here is our home.html:

<html>
<head><title>Home Page</title></head>
<body>
<h1>Hello !</h1>
<p>Welcome to <span th:text="${appName}">Our App</span></p>
</body>
</html>

Note how we used a property we defined in our properties – and then injected that so that we can show it on our home page.

5. Security

Next, let’s add security to our application – by first including the security starter:

<dependency> 
    <groupId>org.springframework.boot</groupId> 
    <artifactId>spring-boot-starter-security</artifactId> 
</dependency>

By now, you’re hopefully noticing a pattern – most Spring libraries are easily imported into our project with the use of simple Boot starters.

Once the spring-boot-starter-security dependency on the classpath of the application – all endpoints are secured by default, using either httpBasic or formLogin based on Spring Security’s content-negotiation strategy.

That’s why, if we have the starter on the classpath, we should usually define our own custom Security configuration by extending the WebSecurityConfigurerAdapter class:

@Configuration
@EnableWebSecurity
public class SecurityConfig extends WebSecurityConfigurerAdapter {

    @Override
    protected void configure(HttpSecurity http) throws Exception {
        http.authorizeRequests()
            .anyRequest()
            .permitAll()
            .and().csrf().disable();
    }
}

In our example, we’re allowing unrestricted access to all endpoints.

Of course, Spring Security is an extensive topic and one not easily covered in a couple of lines of configuration – so I definitely encourage you to go deeper into the topic.

6. Simple Persistence

Let’s start by defining our data model – a simple Book entity:

@Entity
public class Book {
 
    @Id
    @GeneratedValue(strategy = GenerationType.AUTO)
    private long id;

    @Column(nullable = false, unique = true)
    private String title;

    @Column(nullable = false)
    private String author;
}

And its repository, making good use of Spring Data here:

public interface BookRepository extends CrudRepository<Book, Long> {
    List<Book> findByTitle(String title);
}

Finally, we need to of course configure our new persistence layer:

@EnableJpaRepositories("com.baeldung.persistence.repo") 
@EntityScan("com.baeldung.persistence.model")
@SpringBootApplication 
public class Application {
   ...
}

Note that we’re using:

  • @EnableJpaRepositories to scan the specified package for repositories
  • @EntityScan to pick up our JPA entities

To keep things simple, we’re using an H2 in-memory database here – so that we don’t have any external dependencies when we run the project.

Once we include H2 dependency, Spring Boot auto-detects it and sets up our persistence with no need for extra configuration, other than the data source properties:

spring.datasource.driver-class-name=org.h2.Driver
spring.datasource.url=jdbc:h2:mem:bootapp;DB_CLOSE_DELAY=-1
spring.datasource.username=sa
spring.datasource.password=

Of course, like security, persistence is a broader topic than this basic set here, and one you should certainly explore further.

7. Web and the Controller

Next, let’s have a look at a web tier – and we’ll start that by setting up a simple controller – the BookController.

We’ll implement basic CRUD operations exposing Book resources with some simple validation:

@RestController
@RequestMapping("/api/books")
public class BookController {

    @Autowired
    private BookRepository bookRepository;

    @GetMapping
    public Iterable findAll() {
        return bookRepository.findAll();
    }

    @GetMapping("/title/{bookTitle}")
    public List findByTitle(@PathVariable String bookTitle) {
        return bookRepository.findByTitle(bookTitle);
    }

    @GetMapping("/{id}")
    public Book findOne(@PathVariable Long id) {
        return bookRepository.findById(id)
          .orElseThrow(BookNotFoundException::new);
    }

    @PostMapping
    @ResponseStatus(HttpStatus.CREATED)
    public Book create(@RequestBody Book book) {
        return bookRepository.save(book);
    }

    @DeleteMapping("/{id}")
    public void delete(@PathVariable Long id) {
        bookRepository.findById(id)
          .orElseThrow(BookNotFoundException::new);
        bookRepository.deleteById(id);
    }

    @PutMapping("/{id}")
    public Book updateBook(@RequestBody Book book, @PathVariable Long id) {
        if (book.getId() != id) {
          throw new BookIdMismatchException();
        }
        bookRepository.findById(id)
          .orElseThrow(BookNotFoundException::new);
        return bookRepository.save(book);
    }
}

Given this aspect of the application is an API, we made use of the @RestController annotation here – which equivalent to a @Controller along with @ResponseBody – so that each method marshalls the returned resource right to the HTTP response.

Just one note worth pointing out – we’re exposing our Book entity as our external resource here. That’s fine for our simple application here, but in a real-world application, you will likely want to separate these two concepts.

8. Error Handling

Now that the core application is ready to go, let’s focus on a simple centralized error handling mechanism using @ControllerAdvice:

@ControllerAdvice
public class RestExceptionHandler extends ResponseEntityExceptionHandler {

    @ExceptionHandler({ BookNotFoundException.class })
    protected ResponseEntity<Object> handleNotFound(
      Exception ex, WebRequest request) {
        return handleExceptionInternal(ex, "Book not found", 
          new HttpHeaders(), HttpStatus.NOT_FOUND, request);
    }

    @ExceptionHandler({ BookIdMismatchException.class, 
      ConstraintViolationException.class, 
      DataIntegrityViolationException.class })
    public ResponseEntity<Object> handleBadRequest(
      Exception ex, WebRequest request) {
        return handleExceptionInternal(ex, ex.getLocalizedMessage(), 
          new HttpHeaders(), HttpStatus.BAD_REQUEST, request);
    }
}

Beyond the standard exceptions we’re handling here, we’re also using a custom exception:

BookNotFoundException:

public class BookNotFoundException extends RuntimeException {

    public BookNotFoundException(String message, Throwable cause) {
        super(message, cause);
    }
    // ...
}

This should give you an idea of what’s possible with this global exception handling mechanism. If you’d like to see a full implementation, have a look at the in-depth tutorial.

Note that Spring Boot also provides an /error mapping by default. We can customize its view by creating a simple error.html:

<html lang="en">
<head><title>Error Occurred</title></head>
<body>
    <h1>Error Occurred!</h1>    
    <b>[<span th:text="${status}">status</span>]
        <span th:text="${error}">error</span>
    </b>
    <p th:text="${message}">message</p>
</body>
</html>

Like most other aspects in Boot, we can control that with a simple property:

server.error.path=/error2

9. Testing

Finally, let’s test our new Books API.

We can make use of @SpringBootTest to load the application context and verify there are no errors when running the app:

@RunWith(SpringRunner.class)
@SpringBootTest
public class SpringContextTest {

    @Test
    public void contextLoads() {
    }
}

Next, let’s add a JUnit test that verifies the calls to the API we’re written, using RestAssured:

public class SpringBootBootstrapLiveTest {

    private static final String API_ROOT
      = "http://localhost:8081/api/books";

    private Book createRandomBook() {
        Book book = new Book();
        book.setTitle(randomAlphabetic(10));
        book.setAuthor(randomAlphabetic(15));
        return book;
    }

    private String createBookAsUri(Book book) {
        Response response = RestAssured.given()
          .contentType(MediaType.APPLICATION_JSON_VALUE)
          .body(book)
          .post(API_ROOT);
        return API_ROOT + "/" + response.jsonPath().get("id");
    }
}

First, we can try to find books using variant methods:

@Test
public void whenGetAllBooks_thenOK() {
    Response response = RestAssured.get(API_ROOT);
 
    assertEquals(HttpStatus.OK.value(), response.getStatusCode());
}

@Test
public void whenGetBooksByTitle_thenOK() {
    Book book = createRandomBook();
    createBookAsUri(book);
    Response response = RestAssured.get(
      API_ROOT + "/title/" + book.getTitle());
    
    assertEquals(HttpStatus.OK.value(), response.getStatusCode());
    assertTrue(response.as(List.class)
      .size() > 0);
}
@Test
public void whenGetCreatedBookById_thenOK() {
    Book book = createRandomBook();
    String location = createBookAsUri(book);
    Response response = RestAssured.get(location);
    
    assertEquals(HttpStatus.OK.value(), response.getStatusCode());
    assertEquals(book.getTitle(), response.jsonPath()
      .get("title"));
}

@Test
public void whenGetNotExistBookById_thenNotFound() {
    Response response = RestAssured.get(API_ROOT + "/" + randomNumeric(4));
    
    assertEquals(HttpStatus.NOT_FOUND.value(), response.getStatusCode());
}

Next, we’ll test creating a new book:

@Test
public void whenCreateNewBook_thenCreated() {
    Book book = createRandomBook();
    Response response = RestAssured.given()
      .contentType(MediaType.APPLICATION_JSON_VALUE)
      .body(book)
      .post(API_ROOT);
    
    assertEquals(HttpStatus.CREATED.value(), response.getStatusCode());
}

@Test
public void whenInvalidBook_thenError() {
    Book book = createRandomBook();
    book.setAuthor(null);
    Response response = RestAssured.given()
      .contentType(MediaType.APPLICATION_JSON_VALUE)
      .body(book)
      .post(API_ROOT);
    
    assertEquals(HttpStatus.BAD_REQUEST.value(), response.getStatusCode());
}

Update an existing book:

@Test
public void whenUpdateCreatedBook_thenUpdated() {
    Book book = createRandomBook();
    String location = createBookAsUri(book);
    book.setId(Long.parseLong(location.split("api/books/")[1]));
    book.setAuthor("newAuthor");
    Response response = RestAssured.given()
      .contentType(MediaType.APPLICATION_JSON_VALUE)
      .body(book)
      .put(location);
    
    assertEquals(HttpStatus.OK.value(), response.getStatusCode());

    response = RestAssured.get(location);
    
    assertEquals(HttpStatus.OK.value(), response.getStatusCode());
    assertEquals("newAuthor", response.jsonPath()
      .get("author"));
}

And delete a book:

@Test
public void whenDeleteCreatedBook_thenOk() {
    Book book = createRandomBook();
    String location = createBookAsUri(book);
    Response response = RestAssured.delete(location);
    
    assertEquals(HttpStatus.OK.value(), response.getStatusCode());

    response = RestAssured.get(location);
    assertEquals(HttpStatus.NOT_FOUND.value(), response.getStatusCode());
}

10. Conclusion

This was a quick but comprehensive intro to Spring Boot.

We of course barely scratched the surface here – there’s a lot more to this framework that we can cover in a single intro article.

Log4J Asynchronous Logging
03
Mar
2021

Log4J Asynchronous Logging

Asynchronous Logging

Log4j2 Supports Async loggers. These loggers provide a drastic improvement in performance compared to their synchronous counterparts.

Async Loggers internally use a library called Disruptor for asynchronous logging.

We need to include Disruptor dependency for using async loggers. Add the following to your pom.xml file –

<!-- Needed for Async Logging with Log4j 2 -->
<dependency>
    <groupId>com.lmax</groupId>
    <artifactId>disruptor</artifactId>
    <version>3.3.6</version>
</dependency>
Spring Boot Actuator
30
Mar
2021

Spring Boot Actuator

In this Spring boot actuator tutorial, learn about in-built HTTP endpoints available for any boot application for different monitoring and management purposes. Before the spring framework, if we had to introduce this type of monitoring functionality in our applications then we had to manually develop all those components and that too was very specific to our need. But with spring boot we have Actuator module which makes it very easy.

We just need to configure a few things and we are done – all the management and monitoring related information is easily available. Let’s learn to configure Spring boot 2 actuator endpoints.

1. Spring Boot Actuator Module

Spring boot’s module Actuator allows you to monitor and manage application usages in production environment, without coding and configuration for any of them. These monitoring and management information is exposed via REST like endpoint URLs.

1.1. Actuator Maven Dependency

<dependency><groupId>org.springframework.boot</groupId><artifactId>spring-boot-starter-actuator</artifactId></dependency>
dependencies {implementation 'org.springframework.boot:spring-boot-starter-actuator'}

1.2. Important Actuator Endpoints

Most applications exposes endpoints via HTTP, where the ID of the endpoint along with a prefix of /actuator is mapped to a URL. For example, by default, the health endpoint is mapped to /actuator/health.

By default, only /health and /info are exposed via Web APIs. Rest are exposed via JMX. Use management.endpoints.web.exposure.include=* to expose all endpoints through the Web APIs.

management.endpoints.web.exposure.include=* # To expose only selected endpoints#management.endpoints.jmx.exposure.include=health,info,env,beans

Some of important and widely used actuator endpoints are given below:

ENDPOINTUSAGE
/auditeventsReturns all auto-configuration candidates and the reason why they ‘were’ or ‘were not’ applied.
/beansReturns a complete list of all the Spring beans in your application.
/mappingsDisplays a collated list of all @RequestMapping paths..
/envReturns list of properties in current environment
/healthReturns application health information.
/cachesIt exposes available caches.
/conditionsShows the conditions that were evaluated on configuration and auto-configuration.
/configpropsIt displays a collated list of all @ConfigurationProperties.
/integrationgraphIt shows the Spring Integration graph. Requires a dependency on spring-integration-core.
/loggersThe configuration of loggers in the application..
/scheduledtasksDisplays the scheduled tasks in the application.
/sessionsReturns trace logs (by default the last 100 HTTP requests). Requires an HttpTraceRepository bean.
/httptraceIt allows retrieval and deletion of user sessions from a Spring Session-backed session store. Requires a Servlet-based web application using Spring Session.
/shutdownLets the application be gracefully shutdown. Disabled by default.
/threaddumpIt performs a thread dump.
/metricsIt shows several useful metrics information like JVM memory used, system CPU usage, open files, and much more.

The Spring web application (Spring MVC, Spring WebFlux, or Jersey) provide the following additional endpoints:

ENDPOINTUSAGE
/heapdumpReturns an hprof heap dump file.
/logfileReturns the contents of the logfile if logging.file.name or logging.file.path properties have been set.

1.3. Securing Endpoints

By default, Spring Security is enabled for all actuator endpoints if it available in the classpath.

If you wish to configure custom security for HTTP endpoints, for example, only allow users with a certain role to access then configure WebSecurityConfigurerAdapter in following manner:

@Configuration(proxyBeanMethods = false)public class ActuatorSecurity extends WebSecurityConfigurerAdapter { @Overrideprotected void configure(HttpSecurity http) throws Exception {http.requestMatcher(EndpointRequest.toAnyEndpoint()).authorizeRequests((requests) ->requests.anyRequest().hasRole("ENDPOINT_ADMIN"));http.httpBasic();} }

The above configuration ensures that only users with role ENDPOINT_ADMIN have access to actuation endpoints.

1.4. Enabling Endpoints

By default, all endpoints (except /shutdown) are enabled. To disable all endpoints, by default, use property:

management.endpoints.enabled-by-default=false

Then use the only required endpoints which the application need to expose using the pattern management.endpoint.<id>.enabled.

management.endpoint.health.enabled=truemanagement.endpoint.loggers.enabled=true

1.5. CORS support

CORS Support is disabled by default and is only enabled once the endpoints.cors.allowed-origins property has been set.

management.endpoints.web.cors.allowed-origins=https://example.commanagement.endpoints.web.cors.allowed-methods=GET,POST

Here the management context path is /management.

1.6. Caching the Response

Actuator endpoints automatically cache the responses to read operations that do not take any parameters. Use cache.time-to-live property to configure the amount of time for which an endpoint will cache the response.

management.endpoint.beans.cache.time-to-live=20s

2. Spring Boot Actuator Endpoint Example

In this example, we will create a simple spring boot application and access the actuator endpoints to know more about them.

2.1. Development environment

  • JDK 1.8, Eclipse, Maven – Development environment
  • Spring-boot – Underlying application framework
  • Spring-boot Actuator – Management endpoints

2.2. Create Maven Project

Start with creating one spring boot project from Spring Initializer site with WebRest Repositories and Actuator dependencies. Download project in zipped format. Unzip and then import project in eclipse as maven project.

Generate Spring Boot Project
Generate Spring Boot Project

2.3. Add simple Rest endpoint

Now add one simple Rest endpoint /example to the application.

package com.example.springbootmanagementexample; import java.util.Date;import org.springframework.web.bind.annotation.GetMapping;import org.springframework.web.bind.annotation.RestController; @RestControllerpublic class SimpleRestController {@GetMapping("/example")public String example() {return "Hello User !! " + new Date();}}

3. Spring Boot Actuator Endpoints Demo

I have added management.security.enabled=false entry to the application.properties file to disable actuator security. Here I am more interested in actuator endpoints responses.

Do maven build using mvn clean install and start the application using java -jar target\spring-boot-actuator-example-0.0.1-SNAPSHOT.jar command. This will bring up one tomcat server in default port 8080 and application will be deployed in it.

Access /example API in browser to generate few monitoring information on server.

  • http://localhost:8080/actuator/envThis will give all the environmental configuration about the server.Endpoint env outputEndpoint env output
  • http://localhost:8080/actuator/beansThis will give all the spring beans loaded in the context.Endpoint beans outputEndpoint beans output
  • http://localhost:8080/actuator/threaddumpThis will give the current server thread dump.Endpoint threaddump outputEndpoint threaddump output

Those endpoints will give standard information in the browser. These are the basic important endpoints we generally refer, but spring boot provides many more endpoints as mentioned in this link

4. Actuator Advance Configuration Options

4.1. Change the Management endpoint context path

By default all endpoints comes in default context path of the application, suffixed with /actuator. If for some reason, we have existing endpoints in application starting with /actuator then we can customize the base path to something else.

All we need to specify the new base path in the application.properties.

management.endpoints.web.base-path=/manage

Now you will be able to access all actuator endpoints under a new URL. e.g.

  • /manage/health
  • /manage/dump
  • /manage/env
  • /manage/beans

4.2. Customize the management server port

To customize the management endpoint port, we need to add this entry in the application.properties file.

management.server.port=8081

5. Summary

In this spring boot actuator example, we learned to configure management and monitoring endpoints with few easy configurations. So next time, you need to add application health checks or add monitoring support, you should consider adding the Spring actuator project and use these endpoints.

Feel free to drop your questions in the comments section.

Happy Learning !!

Spring Boot CRUD Application
30
Mar
2021

Spring Boot CRUD Application

Learn to build Spring boot web applications supporting CRUD operations through form-based UI built on thymeleaf and spring mvc support.

1. Overview

In this tutorial, we are creating a web application with two views :

  • List all employees view – It display all the employees from database in a tabular format in UI. Additionally, there are links to update or delete any employee. This screen also has a separate option which can navigate to create employee screen.
  • Create/update employee view – This screen is used to add a new employee or edit the details of a existing .Add employee screen

There are two main components in this example to focus – MVC controller and UI views.

2. Spring MVC Controller

The controller class has URL mappings and it’s handler methods. There are handler methods are all CRUD operations including POST operation to handle form submission to create/update an employee.

Notice how given handler methods are binding model data to view; and they return view names in string format which gets resolved by view resolver in HTML files.

import java.util.List;import java.util.Optional; import org.springframework.beans.factory.annotation.Autowired;import org.springframework.stereotype.Controller;import org.springframework.ui.Model;import org.springframework.web.bind.annotation.PathVariable;import org.springframework.web.bind.annotation.RequestMapping;import org.springframework.web.bind.annotation.RequestMethod; import com.fusebes.demo.entity.EmployeeEntity;import com.fusebes.demo.exception.RecordNotFoundException;import com.fusebes.demo.service.EmployeeService; @Controller@RequestMapping("/")public class EmployeeMvcController {@AutowiredEmployeeService service; @RequestMappingpublic String getAllEmployees(Model model) {List<EmployeeEntity> list = service.getAllEmployees(); model.addAttribute("employees", list);return "list-employees";} @RequestMapping(path = {"/edit", "/edit/{id}"})public String editEmployeeById(Model model, @PathVariable("id") Optional<Long> id) throws RecordNotFoundException {if (id.isPresent()) {EmployeeEntity entity = service.getEmployeeById(id.get());model.addAttribute("employee", entity);} else {model.addAttribute("employee", new EmployeeEntity());}return "add-edit-employee";} @RequestMapping(path = "/delete/{id}")public String deleteEmployeeById(Model model, @PathVariable("id") Long id) throws RecordNotFoundException {service.deleteEmployeeById(id);return "redirect:/";} @RequestMapping(path = "/createEmployee", method = RequestMethod.POST)public String createOrUpdateEmployee(EmployeeEntity employee) {service.createOrUpdateEmployee(employee);return "redirect:/";}}
  • getAllEmployees() – It returns list of all employees and is mapped to path “/”. It’s default view of the application.
  • editEmployeeById() – It is used to either add new or edit an existing employee. It uses the same HTML view for both operations. If there is an employee Id in context, then that employee is edited – else a new employee is created.
  • deleteEmployeeById() – A simple URL request to delete an employee by it’s Id.
  • createOrUpdateEmployee() – This method handle the HTTP POST requests which are used to either create a new employee or update an employee. Create or update operation depends on presence of employee id in model.

3. Thymeleaf templates

As stated before we are using two views in this example.

<!DOCTYPE html><html xmlns:th="http://www.thymeleaf.org"> <head><meta charset="utf-8"><meta http-equiv="x-ua-compatible" content="ie=edge"><title>All Employees</title><meta name="viewport" content="width=device-width, initial-scale=1"><link rel="stylesheet" href="https://stackpath.bootstrapcdn.com/bootstrap/4.1.3/css/bootstrap.min.css"><link rel="stylesheet" href="https://use.fontawesome.com/releases/v5.4.1/css/all.css"></head> <body><div class="container my-2"><div class="card"><div class="card-body"><div th:switch="${employees}" class="container my-5"><p class="my-5"><a href="/edit" class="btn btn-primary"><i class="fas fa-user-plus ml-2"> Add Employee </i></a></p><div class="col-md-10"><h2 th:case="null">No record found !!</h2><div th:case="*"><table class="table table-striped table-responsive-md"><thead><tr><th>First Name</th><th>Last Name</th><th>Email</th><th>Edit</th><th>Delete</th></tr></thead><tbody><tr th:each="employee : ${employees}"><td th:text="${employee.firstName}"></td><td th:text="${employee.lastName}"></td><td th:text="${employee.email}"></td><td><a th:href="@{/edit/{id}(id=${employee.id})}"class="btn btn-primary"><i class="fas fa-user-edit ml-2"></i></a></td><td><a th:href="@{/delete/{id}(id=${employee.id})}"class="btn btn-primary"><i class="fas fa-user-times ml-2"></i></a></td></tr></tbody></table></div> </div></div></div></div></div></body> </html>
<!DOCTYPE html><html xmlns:th="http://www.thymeleaf.org"> <head><meta charset="utf-8"><meta http-equiv="x-ua-compatible" content="ie=edge"><title>Add Employee</title><meta name="viewport" content="width=device-width, initial-scale=1"><link rel="stylesheet" href="https://stackpath.bootstrapcdn.com/bootstrap/4.1.3/css/bootstrap.min.css"><link rel="stylesheet" href="https://use.fontawesome.com/releases/v5.4.1/css/all.css"></head> <body><div class="container my-5"><h3> Add Employee</h3><div class="card"><div class="card-body"><div class="col-md-10"><form action="#" th:action="@{/createEmployee}" th:object="${employee}"method="post"><div class="row"><div class="form-group col-md-8"><label for="name" class="col-form-label">First Name</label> <input type="text" th:field="*{firstName}" class="form-control"id="firstName" placeholder="First Name" /></div><div class="form-group col-md-8"><label for="name" class="col-form-label">Last Name</label> <input type="text" th:field="*{lastName}" class="form-control"id="lastName" placeholder="Last Name" /></div><div class="form-group col-md-8"><label for="email" class="col-form-label">Email</label> <input type="text" th:field="*{email}" class="form-control"id="email" placeholder="Email Id" /></div> <div class="col-md-6"><input type="submit" class="btn btn-primary" value=" Submit "></div> <input type="hidden" id="id" th:field="*{id}"> </div></form></div></div></div></div></body> </html>

4. Entity and repository

We have bound the EmployeeEntity class as model to UI.

import javax.persistence.Column;import javax.persistence.Entity;import javax.persistence.GeneratedValue;import javax.persistence.Id;import javax.persistence.Table; @Entity@Table(name="TBL_EMPLOYEES")public class EmployeeEntity { @Id@GeneratedValue(strategy = GenerationType.IDENTITY)private Long id; @Column(name="first_name")private String firstName; @Column(name="last_name")private String lastName; @Column(name="email", nullable=false, length=200)private String email; //Setters and getters @Overridepublic String toString() {return "EmployeeEntity [id=" + id + ", firstName=" + firstName + ", lastName=" + lastName + ", email=" + email   + "]";}}

To persist data in database, we are using H2 (In-Memory) Database and using Spring data’s CrudRepository interface. It provides out of the box in-built method for simple CRUD operations.

import org.springframework.data.repository.CrudRepository;import org.springframework.stereotype.Repository; import com.fusebes.demo.entity.EmployeeEntity; @Repositorypublic interface EmployeeRepository extends CrudRepository<EmployeeEntity, Long> { }

Note that repository is initialized with two SQL files which create the database tables and populate default data to it.

DROP TABLE IF EXISTS TBL_EMPLOYEES; CREATE TABLE TBL_EMPLOYEES (id INT AUTO_INCREMENT  PRIMARY KEY,first_name VARCHAR(250) NOT NULL,last_name VARCHAR(250) NOT NULL,email VARCHAR(250) DEFAULT NULL);
INSERT INTO TBL_EMPLOYEES (first_name, last_name, email) VALUES('Lokesh', 'Gupta', 'fusebes@gmail.com'),('John', 'Doe', 'xyz@email.com');

5. Service class

Another important class to see is EmployeeService class through which controller interacts with repository. It contains additional business logic to perform.

import java.util.ArrayList;import java.util.List;import java.util.Optional; import org.springframework.beans.factory.annotation.Autowired;import org.springframework.stereotype.Service; import com.fusebes.demo.entity.EmployeeEntity;import com.fusebesemo.exception.RecordNotFoundException;import com.fusebes.demo.repository.EmployeeRepository; @Servicepublic class EmployeeService { @AutowiredEmployeeRepository repository; public List<EmployeeEntity> getAllEmployees(){List<EmployeeEntity> result = (List<EmployeeEntity>) repository.findAll(); if(result.size() > 0) {return result;} else {return new ArrayList<EmployeeEntity>();}} public EmployeeEntity getEmployeeById(Long id) throws RecordNotFoundException {Optional<EmployeeEntity> employee = repository.findById(id); if(employee.isPresent()) {return employee.get();} else {throw new RecordNotFoundException("No employee record exist for given id");}} public EmployeeEntity createOrUpdateEmployee(EmployeeEntity entity) {if(entity.getId()  == null) {entity = repository.save(entity); return entity;} else{Optional<EmployeeEntity> employee = repository.findById(entity.getId()); if(employee.isPresent()) {EmployeeEntity newEntity = employee.get();newEntity.setEmail(entity.getEmail());newEntity.setFirstName(entity.getFirstName());newEntity.setLastName(entity.getLastName()); newEntity = repository.save(newEntity); return newEntity;} else {entity = repository.save(entity); return entity;}}}  public void deleteEmployeeById(Long id) throws RecordNotFoundException {Optional<EmployeeEntity> employee = repository.findById(id); if(employee.isPresent()) {repository.deleteById(id);} else {throw new RecordNotFoundException("No employee record exist for given id");}} }

6. Add Spring boot and Thymeleaf maven dependencies

In spring boot projects, we only need to add spring-boot-starter-thymeleaf dependency and project itself auto-configures thymeleaf with default configuration. It read the HTML templates from /src/main/resources/templates folder.

<?xml version="1.0" encoding="UTF-8"?><project xmlns="http://maven.apache.org/POM/4.0.0"xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd"><modelVersion>4.0.0</modelVersion><parent><groupId>org.springframework.boot</groupId><artifactId>spring-boot-starter-parent</artifactId><version>2.1.5.RELEASE</version><relativePath /> <!-- lookup parent from repository --></parent><groupId>com.fusebes</groupId><artifactId>demo</artifactId><version>0.0.1-SNAPSHOT</version><name>demo</name><description>Demo project for Spring Boot</description> <properties><java.version>1.8</java.version></properties> <dependencies><dependency><groupId>org.springframework.boot</groupId><artifactId>spring-boot-starter-web</artifactId></dependency><dependency><groupId>org.springframework.boot</groupId><artifactId>spring-boot-starter-data-jpa</artifactId></dependency><dependency><groupId>org.springframework.boot</groupId><artifactId>spring-boot-starter-thymeleaf</artifactId></dependency><dependency><groupId>com.h2database</groupId><artifactId>h2</artifactId><scope>runtime</scope></dependency><dependency><groupId>org.springframework.boot</groupId><artifactId>spring-boot-starter-test</artifactId><scope>test</scope></dependency></dependencies></project>

7. spring boot thymeleaf crud tutorial demo

Start this application as Spring boot application which starts the web application in the embedded tomcat server.

Hit the URL : http://localhost:8080/

Verify that screen is rendered with two default employee details from data.sql file.

Play with application. Create few new employees, edit existing employees. Delete some employees.

Spring Boot2 @SpringBootApplication Auto Configuration
30
Mar
2021

Spring Boot 2 @SpringBootApplication Auto Configuration

Spring boot is very easy to use and it does a lot of things under the hood, you might not be aware of. In future, a good developer will be who will know exactly what is going on behind spring boot auto configuration, how to use it in your favor and how to disable certain sections which you do not want into your project.

To understand most basic things behind spring boot, we will create a minimum boot application with single dependency and single launch class file. We will then analyze the startup logs to get the insights.

Create Spring boot application with launch class

  1. Create a new maven project in eclipse with archetype “maven-archetype-quickstart“.
  2. Update pom.xml file with spring-boot-starter-web dependency and plugin information.<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd;<modelVersion>4.0.0</modelVersion> <groupId>com.fusebes</groupId><artifactId>springbootdemo</artifactId><version>0.0.1-SNAPSHOT</version><packaging>jar</packaging> <name>springbootdemo</name><url>http://maven.apache.org</url> <parent><groupId>org.springframework.boot</groupId><artifactId>spring-boot-starter-parent</artifactId><version>2.0.0.RELEASE</version></parent> <properties><java.version>1.8</java.version><project.build.sourceEncoding>UTF-8</project.build.sourceEncoding></properties> <dependencies><dependency><groupId>org.springframework.boot</groupId><artifactId>spring-boot-starter-web</artifactId></dependency></dependencies> <build><plugins><plugin><groupId>org.springframework.boot</groupId><artifactId>spring-boot-maven-plugin</artifactId></plugin></plugins></build> <repositories><repository><id>repository.spring.release</id><name>Spring GA Repository</name><url>http://repo.spring.io/release</url></repository></repositories></project>
  3. Create launch application.package com.fusebes.demo; import org.springframework.boot.SpringApplication;import org.springframework.boot.autoconfigure.SpringBootApplication;import org.springframework.context.ApplicationContext; @SpringBootApplicationpublic class App {public static void main(String[] args) {ApplicationContext ctx = SpringApplication.run(App.class, args);}}What this launch class does?Above class is called spring boot application launch class. It used to Bootstrap and launch a Spring application from a Java main() method. It typically does following things –
    • Create an instance of Spring’s ApplicationContext.
    • Enable the functionality to accept command-line arguments and expose them as Spring properties.
    • Load all the Spring beans as per the configuration. You can do other operations as well as per project need arises.

@SpringBootApplication Annotation

This annotation is a shortcut of applying 3 annotations in one statement –

  1. @SpringBootConfiguration@SpringBootConfiguration is new annotation in Spring boot 2. Previously, we have been using @Configuration annotation. You can use @Configuration in place of this. Both are same thing.It indicates that a class provides Spring Boot application @Configuration. It simply means that annotated class is a configuration class and shall be scanned for further configurations and bean definitions.
  2. @EnableAutoConfigurationThis annotation is used to enable auto-configuration of the Spring Application Context, attempting to guess and configure beans that you are likely to need. Auto-configuration classes are usually applied based on your classpath and what beans you have defined.Auto-configuration tries to be as intelligent as possible and will back-away as you define more of your own configuration. You can always manually exclude any configuration that you never want to apply using two methods –i) Use excludeName()
    ii) Using the spring.autoconfigure.exclude property in properties file. e.g.@EnableAutoConfiguration(excludeName = {"multipartResolver","mbeanServer"})Auto-configuration is always applied after user-defined beans have been registered.
  3. @ComponentScanThis annotation provides support parallel with Spring XML’s context:component-scan element.Either basePackageClasses() or basePackages() may be specified to define specific packages to scan. If specific packages are not defined, scanning will occur from the package of the class that declares this annotation.

Run the launch application and check logs

Let’s start running it with the simplest option–running as a Java application. In your IDE, right-click on the application class and run it as Java Application. For getting insight of registered beans, I have added modified the launch application as below.

@SpringBootApplicationpublic class App {public static void main(String[] args) {ApplicationContext ctx = SpringApplication.run(App.class, args); String[] beanNames = ctx.getBeanDefinitionNames(); Arrays.sort(beanNames); for (String beanName : beanNames) {System.out.println(beanName);}}}

Now see the logs –

.   ____          _            __ _ _/\\ / ___'_ __ _ _(_)_ __  __ _ \ \ \ \( ( )\___ | '_ | '_| | '_ \/ _` | \ \ \ \\\/  ___)| |_)| | | | | || (_| |  ) ) ) )'  |____| .__|_| |_|_| |_\__, | / / / /=========|_|==============|___/=/_/_/_/:: Spring Boot ::        (v2.0.0.RELEASE) 2018-04-02 13:09:41.100  INFO 11452 --- [           main] com.fusebes.demo.App               : Starting App on FFC15B4E9C5AA with PID 11452 (C:\Users\zkpkhua\IDPPaymentTransfers_Integrated\springbootdemo\target\classes started by zkpkhua in C:\Users\zkpkhua\IDPPaymentTransfers_Integrated\springbootdemo)2018-04-02 13:09:41.108  INFO 11452 --- [           main] com.fusebes.demo.App               : No active profile set, falling back to default profiles: default2018-04-02 13:09:41.222  INFO 11452 --- [           main] ConfigServletWebServerApplicationContext : Refreshing org.springframework.boot.web.servlet.context.AnnotationConfigServletWebServerApplicationContext@4450d156: startup date [Mon Apr 02 13:09:41 IST 2018]; root of context hierarchy2018-04-02 13:09:43.474  INFO 11452 --- [           main] o.s.b.w.embedded.tomcat.TomcatWebServer  : Tomcat initialized with port(s): 8080 (http)2018-04-02 13:09:43.526  INFO 11452 --- [           main] o.apache.catalina.core.StandardService   : Starting service [Tomcat]2018-04-02 13:09:43.526  INFO 11452 --- [           main] org.apache.catalina.core.StandardEngine  : Starting Servlet Engine: Apache Tomcat/8.5.282018-04-02 13:09:43.748  INFO 11452 --- [ost-startStop-1] o.a.c.c.C.[Tomcat].[localhost].[/]       : Initializing Spring embedded WebApplicationContext2018-04-02 13:09:43.748  INFO 11452 --- [ost-startStop-1] o.s.web.context.ContextLoader            : Root WebApplicationContext: initialization completed in 2531 ms2018-04-02 13:09:43.964  INFO 11452 --- [ost-startStop-1] o.s.b.w.servlet.ServletRegistrationBean  : Servlet dispatcherServlet mapped to [/]2018-04-02 13:09:43.969  INFO 11452 --- [ost-startStop-1] o.s.b.w.servlet.FilterRegistrationBean   : Mapping filter: 'characterEncodingFilter' to: [/*]2018-04-02 13:09:43.970  INFO 11452 --- [ost-startStop-1] o.s.b.w.servlet.FilterRegistrationBean   : Mapping filter: 'hiddenHttpMethodFilter' to: [/*]2018-04-02 13:09:43.970  INFO 11452 --- [ost-startStop-1] o.s.b.w.servlet.FilterRegistrationBean   : Mapping filter: 'httpPutFormContentFilter' to: [/*]2018-04-02 13:09:43.970  INFO 11452 --- [ost-startStop-1] o.s.b.w.servlet.FilterRegistrationBean   : Mapping filter: 'requestContextFilter' to: [/*]2018-04-02 13:09:44.480  INFO 11452 --- [           main] s.w.s.m.m.a.RequestMappingHandlerAdapter : Looking for @ControllerAdvice: org.springframework.boot.web.servlet.context.AnnotationConfigServletWebServerApplicationContext@4450d156: startup date [Mon Apr 02 13:09:41 IST 2018]; root of context hierarchy2018-04-02 13:09:44.627  INFO 11452 --- [           main] s.w.s.m.m.a.RequestMappingHandlerMapping : Mapped "{[/error]}" onto public org.springframework.http.ResponseEntity<java.util.Map<java.lang.String, java.lang.Object>> org.springframework.boot.autoconfigure.web.servlet.error.BasicErrorController.error(javax.servlet.http.HttpServletRequest)2018-04-02 13:09:44.630  INFO 11452 --- [           main] s.w.s.m.m.a.RequestMappingHandlerMapping : Mapped "{[/error],produces=}" onto public org.springframework.web.servlet.ModelAndView org.springframework.boot.autoconfigure.web.servlet.error.BasicErrorController.errorHtml(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse)2018-04-02 13:09:44.681  INFO 11452 --- [           main] o.s.w.s.handler.SimpleUrlHandlerMapping  : Mapped URL path [/webjars/**] onto handler of type [class org.springframework.web.servlet.resource.ResourceHttpRequestHandler]2018-04-02 13:09:44.682  INFO 11452 --- [           main] o.s.w.s.handler.SimpleUrlHandlerMapping  : Mapped URL path [/**] onto handler of type [class org.springframework.web.servlet.resource.ResourceHttpRequestHandler]2018-04-02 13:09:44.747  INFO 11452 --- [           main] o.s.w.s.handler.SimpleUrlHandlerMapping  : Mapped URL path [/**/favicon.ico] onto handler of type [class org.springframework.web.servlet.resource.ResourceHttpRequestHandler]2018-04-02 13:09:45.002  INFO 11452 --- [           main] o.s.j.e.a.AnnotationMBeanExporter        : Registering beans for JMX exposure on startup2018-04-02 13:09:45.070  INFO 11452 --- [           main] o.s.b.w.embedded.tomcat.TomcatWebServer  : Tomcat started on port(s): 8080 (http) with context path ''2018-04-02 13:09:45.076  INFO 11452 --- [           main] com.fusebes.demo.App               : Started App in 4.609 seconds (JVM running for 5.263)appbasicErrorControllerbeanNameHandlerMappingbeanNameViewResolvercharacterEncodingFilterconventionErrorViewResolverdefaultServletHandlerMappingdefaultValidatordefaultViewResolverdispatcherServletdispatcherServletRegistrationerrorerrorAttributeserrorPageCustomizererrorPageRegistrarBeanPostProcessorfaviconHandlerMappingfaviconRequestHandlerhandlerExceptionResolverhiddenHttpMethodFilterhttpPutFormContentFilterhttpRequestHandlerAdapterjacksonCodecCustomizerjacksonObjectMapperjacksonObjectMapperBuilderjsonComponentModulelocaleCharsetMappingsCustomizermappingJackson2HttpMessageConvertermbeanExportermbeanServermessageConvertersmethodValidationPostProcessormultipartConfigElementmultipartResolvermvcContentNegotiationManagermvcConversionServicemvcHandlerMappingIntrospectormvcPathMatchermvcResourceUrlProvidermvcUriComponentsContributormvcUrlPathHelpermvcValidatormvcViewResolverobjectNamingStrategyorg.springframework.boot.autoconfigure.AutoConfigurationPackagesorg.springframework.boot.autoconfigure.condition.BeanTypeRegistryorg.springframework.boot.autoconfigure.context.ConfigurationPropertiesAutoConfigurationorg.springframework.boot.autoconfigure.context.PropertyPlaceholderAutoConfigurationorg.springframework.boot.autoconfigure.http.HttpMessageConvertersAutoConfigurationorg.springframework.boot.autoconfigure.http.HttpMessageConvertersAutoConfiguration$StringHttpMessageConverterConfigurationorg.springframework.boot.autoconfigure.http.JacksonHttpMessageConvertersConfigurationorg.springframework.boot.autoconfigure.http.JacksonHttpMessageConvertersConfiguration$MappingJackson2HttpMessageConverterConfigurationorg.springframework.boot.autoconfigure.http.codec.CodecsAutoConfigurationorg.springframework.boot.autoconfigure.http.codec.CodecsAutoConfiguration$JacksonCodecConfigurationorg.springframework.boot.autoconfigure.info.ProjectInfoAutoConfigurationorg.springframework.boot.autoconfigure.internalCachingMetadataReaderFactoryorg.springframework.boot.autoconfigure.jackson.JacksonAutoConfigurationorg.springframework.boot.autoconfigure.jackson.JacksonAutoConfiguration$Jackson2ObjectMapperBuilderCustomizerConfigurationorg.springframework.boot.autoconfigure.jackson.JacksonAutoConfiguration$JacksonObjectMapperBuilderConfigurationorg.springframework.boot.autoconfigure.jackson.JacksonAutoConfiguration$JacksonObjectMapperConfigurationorg.springframework.boot.autoconfigure.jackson.JacksonAutoConfiguration$ParameterNamesModuleConfigurationorg.springframework.boot.autoconfigure.jmx.JmxAutoConfigurationorg.springframework.boot.autoconfigure.security.reactive.ReactiveSecurityAutoConfigurationorg.springframework.boot.autoconfigure.validation.ValidationAutoConfigurationorg.springframework.boot.autoconfigure.web.client.RestTemplateAutoConfigurationorg.springframework.boot.autoconfigure.web.embedded.EmbeddedWebServerFactoryCustomizerAutoConfigurationorg.springframework.boot.autoconfigure.web.embedded.EmbeddedWebServerFactoryCustomizerAutoConfiguration$TomcatWebServerFactoryCustomizerConfigurationorg.springframework.boot.autoconfigure.web.servlet.DispatcherServletAutoConfigurationorg.springframework.boot.autoconfigure.web.servlet.DispatcherServletAutoConfiguration$DispatcherServletConfigurationorg.springframework.boot.autoconfigure.web.servlet.DispatcherServletAutoConfiguration$DispatcherServletRegistrationConfigurationorg.springframework.boot.autoconfigure.web.servlet.HttpEncodingAutoConfigurationorg.springframework.boot.autoconfigure.web.servlet.MultipartAutoConfigurationorg.springframework.boot.autoconfigure.web.servlet.ServletWebServerFactoryAutoConfigurationorg.springframework.boot.autoconfigure.web.servlet.ServletWebServerFactoryConfiguration$EmbeddedTomcatorg.springframework.boot.autoconfigure.web.servlet.WebMvcAutoConfigurationorg.springframework.boot.autoconfigure.web.servlet.WebMvcAutoConfiguration$EnableWebMvcConfigurationorg.springframework.boot.autoconfigure.web.servlet.WebMvcAutoConfiguration$WebMvcAutoConfigurationAdapterorg.springframework.boot.autoconfigure.web.servlet.WebMvcAutoConfiguration$WebMvcAutoConfigurationAdapter$FaviconConfigurationorg.springframework.boot.autoconfigure.web.servlet.error.ErrorMvcAutoConfigurationorg.springframework.boot.autoconfigure.web.servlet.error.ErrorMvcAutoConfiguration$DefaultErrorViewResolverConfigurationorg.springframework.boot.autoconfigure.web.servlet.error.ErrorMvcAutoConfiguration$WhitelabelErrorViewConfigurationorg.springframework.boot.autoconfigure.websocket.servlet.WebSocketServletAutoConfigurationorg.springframework.boot.autoconfigure.websocket.servlet.WebSocketServletAutoConfiguration$TomcatWebSocketConfigurationorg.springframework.boot.context.properties.ConfigurationBeanFactoryMetadataorg.springframework.boot.context.properties.ConfigurationPropertiesBindingPostProcessororg.springframework.context.annotation.internalAutowiredAnnotationProcessororg.springframework.context.annotation.internalCommonAnnotationProcessororg.springframework.context.annotation.internalConfigurationAnnotationProcessororg.springframework.context.annotation.internalRequiredAnnotationProcessororg.springframework.context.event.internalEventListenerFactoryorg.springframework.context.event.internalEventListenerProcessorparameterNamesModulepreserveErrorControllerTargetClassPostProcessorpropertySourcesPlaceholderConfigurerrequestContextFilterrequestMappingHandlerAdapterrequestMappingHandlerMappingresourceHandlerMappingrestTemplateBuilderserver-org.springframework.boot.autoconfigure.web.ServerPropertiesservletWebServerFactoryCustomizersimpleControllerHandlerAdapterspring.http.encoding-org.springframework.boot.autoconfigure.http.HttpEncodingPropertiesspring.info-org.springframework.boot.autoconfigure.info.ProjectInfoPropertiesspring.jackson-org.springframework.boot.autoconfigure.jackson.JacksonPropertiesspring.mvc-org.springframework.boot.autoconfigure.web.servlet.WebMvcPropertiesspring.resources-org.springframework.boot.autoconfigure.web.ResourcePropertiesspring.security-org.springframework.boot.autoconfigure.security.SecurityPropertiesspring.servlet.multipart-org.springframework.boot.autoconfigure.web.servlet.MultipartPropertiesstandardJacksonObjectMapperBuilderCustomizerstringHttpMessageConvertertomcatServletWebServerFactorytomcatServletWebServerFactoryCustomizertomcatWebServerFactoryCustomizerviewControllerHandlerMappingviewResolverwebServerFactoryCustomizerBeanPostProcessorwebsocketContainerCustomizerwelcomePageHandlerMapping

You see how many beans got registered automatically. That’s beauty of spring boot. If you want to dig deeper into why any particular bean got registered? You can see that by putting a debug flag at application startup.

Simply pass -Ddebug=true as VM argument.

Now when you run the application, you will get lots of debug logs having similar information :

CodecsAutoConfiguration.JacksonCodecConfiguration matched:- @ConditionalOnClass found required class 'com.fasterxml.jackson.databind.ObjectMapper'; @ConditionalOnMissingClass did not find unwanted class (OnClassCondition) CodecsAutoConfiguration.JacksonCodecConfiguration#jacksonCodecCustomizer matched:- @ConditionalOnBean (types: com.fasterxml.jackson.databind.ObjectMapper; SearchStrategy: all) found bean 'jacksonObjectMapper' (OnBeanCondition) DispatcherServletAutoConfiguration.DispatcherServletConfiguration matched:- @ConditionalOnClass found required class 'javax.servlet.ServletRegistration'; @ConditionalOnMissingClass did not find unwanted class (OnClassCondition)- Default DispatcherServlet did not find dispatcher servlet beans (DispatcherServletAutoConfiguration.DefaultDispatcherServletCondition) DispatcherServletAutoConfiguration.DispatcherServletRegistrationConfiguration matched:- @ConditionalOnClass found required class 'javax.servlet.ServletRegistration'; @ConditionalOnMissingClass did not find unwanted class (OnClassCondition)- DispatcherServlet Registration did not find servlet registration bean (DispatcherServletAutoConfiguration.DispatcherServletRegistrationCondition) .........

Above logs tell why a particular bean was registered into spring context. This information is very useful when you debug the issues with auto configutation.

Similarily, everytime we add a new dependency to a Spring Boot project, Spring Boot auto-configuration automatically tries to configure the beans based on the dependency.

I hope that information discussed above will help you in future while debugging spring boot related issues.

Happy Learning !!

Spring Boot Starter Maven Templates
30
Mar
2021

Spring Boot Starter Maven Templates

Not very long ago, with the exponential increase in number of libraries and their dependencies, dependency management was becoming very complex task which required good amount of technical expertise to do it correctly. With the introduction of String boot starter templates, you can get a lot of help in identifying the correct dependencies to use in project if you want to use any popular library into your project.

Spring Boot comes with over 50+ different starter modules, which provide ready-to-use integration libraries for many different frameworks, such as database connections that are both relational and NoSQL, web services, social network integration, monitoring libraries, logging, template rendering, and the list just keeps going on.

How starter template work?

Spring Boot starters are templates that contain a collection of all the relevant transitive dependencies that are needed to start a particular functionality. Each starter has a special file, which contains the list of all the provided dependencies Spring provides.

These files can be found inside pom.xml files in respective starter module. e.g.

The spring-boot-starter-data-jpa starter pom file can be found at github.

This tells us that by including spring-boot-starter-data-jpa in our build as a dependency, we will automatically get spring-ormhibernate-entity-manager and spring-data-jpa. These libraries will provide us all basic things to start writing JPA/DAO code .

So next time when you want to give your project any specific functionality, I will suggest to check for existing starter templates to see if you can use it directly. Ongoing community additions are always on, so this list is already growing and you can contribute to it as well.

Popular templates and their transitive dependencies

I am listing down some very frequently use spring starters and what dependencies they bring along, for information only.

STARTERDEPENDENCIES
spring-boot-starterspring-boot, spring-context, spring-beans
spring-boot-starter-jerseyjersey-container-servlet-core, jersey-container-servlet, jersey-server
spring-boot-starter-actuatorspring-boot-actuator, micrometer-core
spring-boot-starter-aopspring-aop, aspectjrt, aspectjweaver
spring-boot-starter-data-restspring-hateoas, spring-data-rest-webmvc
spring-boot-starter-hateoasspring-hateoas
spring-boot-starter-logginglogback-classic, jcl-over-slf4j, jul-to-slf4j
spring-boot-starter-log4j2log4j2, log4j-slf4j-impl
spring-boot-starter-securityspring-security-web, spring-security-config
spring-boot-starter-testspring-test, spring-boot,junit,mockito, hamcrest-library, assertj, jsonassert, json-path
spring-boot-starter-web-servicesspring-ws-core

Drop me your questions in comments section.

Happy Learning !!

References:

Spring boot starters
Using boot starters

@Scope – How to get Scope of Bean
14
May
2021

@Scope – How to get Scope of Bean

@Scope – How to get Scope of Bean from Code

When we create a Bean we are creating  actual instances of the class defined by that bean definition. We can also control the scope of the objects created from a particular bean definition.

There are 5 types of scopes in bean,

  • singleton (default scope)
  • prototype
  • request
  • session
  • global-session

Singleton:
Single instance per spring IoC container

Prototype:
Single bean definition to any number of object instances.

Request:
Bean definition for each request. Only valid web-aware Spring ApplicationContext.

Session:
Bean definition for a session. Only valid web-aware Spring ApplicationContext.

Global-Session:
Similar to session but the only makes sense in the context of portlet-based web applications. Only valid web-aware Spring ApplicationContext.

Spring Boot JSP View Resolver Example
30
Mar
2021

Spring Boot JSP View Resolver Example

Learn to create and configure spring boot jsp view resolver which uses JSP template files to render view layer. This example uses embedded Tomcat server to run the application.

Maven dependencies – pom.xml

This application uses given below dependencies.

<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/maven-v4_0_0.xsd"><modelVersion>4.0.0</modelVersion><groupId>com.fusebes</groupId><artifactId>spring-boot-demo</artifactId><packaging>war</packaging><version>0.0.1-SNAPSHOT</version><name>spring-boot-demo Maven Webapp</name><url>http://maven.apache.org</url><parent><groupId>org.springframework.boot</groupId><artifactId>spring-boot-starter-parent</artifactId><version>1.5.1.RELEASE</version></parent><properties><java.version>1.8</java.version></properties><dependencies><!-- Web --><dependency><groupId>org.springframework.boot</groupId><artifactId>spring-boot-starter-web</artifactId></dependency><!-- Tomcat Embed --><dependency><groupId>org.springframework.boot</groupId><artifactId>spring-boot-starter-tomcat</artifactId><scope>provided</scope></dependency><!-- JSTL --><dependency><groupId>javax.servlet</groupId><artifactId>jstl</artifactId></dependency><!-- To compile JSP files --><dependency><groupId>org.apache.tomcat.embed</groupId><artifactId>tomcat-embed-jasper</artifactId><scope>provided</scope></dependency></dependencies></project>

Spring Boot Application Initializer

The first step in producing a deployable war file is to provide a SpringBootServletInitializer subclass and override its configure() method. This makes use of Spring Framework’s Servlet 3.0 support and allows you to configure your application when it’s launched by the servlet container.

package com.fusebes.app.controller; import org.springframework.boot.SpringApplication;import org.springframework.boot.autoconfigure.SpringBootApplication;import org.springframework.boot.builder.SpringApplicationBuilder;import org.springframework.boot.web.support.SpringBootServletInitializer; @SpringBootApplicationpublic class SpringBootWebApplication extends SpringBootServletInitializer { @Overrideprotected SpringApplicationBuilder configure(SpringApplicationBuilder application) {return application.sources(SpringBootWebApplication.class);} public static void main(String[] args) throws Exception {SpringApplication.run(SpringBootWebApplication.class, args);}}

Spring Controller

Controller classes can have methods mapped to specific URLs in the application. In given application, it has two views i.e. “/” and “/next”.

package com.fusebes.app.controller; import java.util.Map; import org.springframework.stereotype.Controller;import org.springframework.web.bind.annotation.RequestMapping; @Controllerpublic class IndexController { @RequestMapping("/")public String home(Map<String, Object> model) {model.put("message", "Fusebes Reader !!");return "index";} @RequestMapping("/next")public String next(Map<String, Object> model) {model.put("message", "You are in new page !!");return "next";} }

Spring Boot JSP ViewResolver Configuration

To resolve JSP files location, you can have two approaches.

1) Add entries in application.properties

spring.mvc.view.prefix=/WEB-INF/view/spring.mvc.view.suffix=.jsp //For detailed logging during development logging.level.org.springframework=TRACElogging.level.com=TRACE

2) Configure InternalResourceViewResolver to serve JSP pages

package com.fusebes.app.controller; import org.springframework.context.annotation.ComponentScan;import org.springframework.context.annotation.Configuration;import org.springframework.web.servlet.config.annotation.EnableWebMvc;import org.springframework.web.servlet.config.annotation.ViewResolverRegistry;import org.springframework.web.servlet.config.annotation.WebMvcConfigurerAdapter;import org.springframework.web.servlet.view.InternalResourceViewResolver;import org.springframework.web.servlet.view.JstlView; @Configuration@EnableWebMvc@ComponentScanpublic class MvcConfiguration extends WebMvcConfigurerAdapter{@Overridepublic void configureViewResolvers(ViewResolverRegistry registry) {InternalResourceViewResolver resolver = new InternalResourceViewResolver();resolver.setPrefix("/WEB-INF/view/");resolver.setSuffix(".jsp");resolver.setViewClass(JstlView.class);registry.viewResolver(resolver);}}

JSP Files

Two used JSP files in this spring boot jsp example – are below.

index.jsp

<!DOCTYPE html><%@ taglib prefix="spring" uri="http://www.springframework.org/tags"%><html lang="en"><body><div><div><h1>Spring Boot JSP Example</h1><h2>Hello ${message}</h2> Click on this <strong><a href="next">link</a></strong> to visit another page.</div></div></body></html>

next.jsp

<!DOCTYPE html><%@ taglib prefix="spring" uri="http://www.springframework.org/tags"%><html lang="en"><body><div><div><h1>Another page</h1><h2>Hello ${message}</h2> Click on this <strong><a href="/">link</a></strong> to visit previous page.</div></div></body></html>

Demo

After whole code is written and placed inside folders, run the application by executing main() method in SpringBootWebApplication class.

Now hit the URL: http://localhost:8080/

And Click next link

Configuring Spring Boot's Server, GZip compression, HTTP/2, caching and much more
03
Mar
2021

Configuring Spring Boot’s Server, GZip compression, HTTP/2, caching and much more

Spring Boot is powerful yet flexible. It tries to auto-configure most of the stuff for you so that you can get up and running quickly with your application.

It uses sensible defaults during auto-configuration but also gives you the flexibility to change those defaults by just tweaking a few things.

In this article, you’ll learn about the most common configuration tweaks that might need to do in your application.

Changing the embedded server in Spring Boot

Spring Boot uses Tomcat as the default embedded server. If you wanna use some other popular server like Jetty or Undertow then you just need to exclude tomcat dependency and add the other server dependency.

1. Using Jetty as the embedded server in Spring Boot

<!-- Exclude tomcat dependency -->
<dependency>
	<groupId>org.springframework.boot</groupId>
	<artifactId>spring-boot-starter-web</artifactId>
	<exclusions>
		<exclusion>
			<groupId>org.springframework.boot</groupId>
			<artifactId>spring-boot-starter-tomcat</artifactId>
		</exclusion>
	</exclusions>
</dependency>
<!-- Include jetty dependency -->
<dependency>
	<groupId>org.springframework.boot</groupId>
	<artifactId>spring-boot-starter-jetty</artifactId>
</dependency>

2. Using undertow as the embedded server in Spring Boot

<!-- Exclude tomcat dependency -->
<dependency>
    <groupId>org.springframework.boot</groupId>
    <artifactId>spring-boot-starter-web</artifactId>
    <exclusions>
        <exclusion>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-tomcat</artifactId>
        </exclusion>
    </exclusions>
</dependency>
<!-- Include undertow dependency -->
<dependency>
    <groupId>org.springframework.boot</groupId>
    <artifactId>spring-boot-starter-undertow</artifactId>
</dependency>

Changing the default server port and context path in Spring Boot

By default, Spring Boot runs your application on port 8080 with the context path /.

If you wanna change the default port and context path, then it’s just a matter of specifying the corresponding values in the application.properties file –

# HTTP Server port
server.port=8080

# Make the application accessible on the given context path (http://localhost:8080/myapp)
server.servlet.context-path=/myapp

Enabling GZip compression in Spring Boot

GZip compression is a very simple and effective way to save bandwidth and improve the speed of your website.

It reduces the response time of your website by compressing the resources and then sending it over to the clients. It saves bandwidth by at least 50%.

GZip compression is disabled by default in Spring Boot. To enable it, add the following properties to your application.properties file –

# Enable response compression
server.compression.enabled=true

# The comma-separated list of mime types that should be compressed
server.compression.mime-types=text/html,text/xml,text/plain,text/css,text/javascript,application/javascript,application/json

# Compress the response only if the response size is at least 1KB
server.compression.min-response-size=1024

Note that, GZip compression has a small overhead. Therefore I’ve added a min-response-size property to tell spring boot server to compress the response only if the size is more than the given value.

Enabling HTTP/2 support in Spring Boot

HTTP/2 is an improvement over the HTTP1 protocol. It improves the page load speed of your website by employing several mechanisms like data compression, server push, multiplexing of multiple requests over a single TCP connection etc.

You can enable HTTP2 in spring boot with the following property, if the server has support for it –

# Enable HTTP/2 support, if the current environment supports it
server.http2.enabled=true

Enabling browser caching of static resources in Spring Boot

Browser caching is another way to improve the page load speed of your website. You can set cache-control headers to tell browsers to cache static resources until a certain period of time.

Browser caching is disabled by default in Spring Boot. You can enable caching by setting the following properties in the application.properties file.

# Maximum time the response should be cached (in seconds) 
spring.resources.cache.cachecontrol.max-age=120

# The cache must re-validate stale resources with the server. Any expired resources must not be used without re-validating.
spring.resources.cache.cachecontrol.must-revalidate=true

Following are few other cache related properties that you should be aware of –

# The resources are private and intended for a single user. They must not be stored by a shared cache (e.g CDN).
spring.resources.cache.cachecontrol.cache-private= # set a boolean value true/false

# The resources are public and any cache may store the response.
spring.resources.cache.cachecontrol.cache-public= # set a boolean value true/false

Configuring multipart file uploads in Spring Boot

Multipart file uploads are enabled by default in Spring Boot with the following property –

spring.servlet.multipart.enabled=true

But there are few other default multipart properties that you might need to change.

By default, Spring Boot allows you to upload a file with a maximum size of 1MB. You might need to change this to the desired value as per your requirements.

Following is the complete set of properties –

# Write files to disk if the file size is more than 2KB.
spring.servlet.multipart.file-size-threshold=2KB

# The intermediate disk location where the uploaded files are written
spring.servlet.multipart.location=/tmp

# Maximum file size that can be uploaded
spring.servlet.multipart.max-file-size=50MB

# Maximum allowed multipart request size
spring.servlet.multipart.max-request-size=75MB

Conclusion

In this short article, you learned how to tweak Spring Boot’s default configurations as per your needs. You can find a full index of common application properties on the following official Spring Boot page –

Spring Boot Common application properties

Whenever you need to configure anything in your spring boot application, the first thing you should do is – check the above common application properties page. Most probably, you’ll find a property for configuring the stuff that you’re looking for.

I hope you enjoyed this article. As always, Thanks for reading.

Why Kafka Is so Fast
26
Mar
2021

Why Kafka Is so Fast

Discover the deliberate design decisions that have made Kafka the performance powerhouse it is today.

The last few years have brought about immense changes in the software architecture landscape. The notion of a single monolithic application or even several coarse-grained services sharing a common data store has been all but erased from the hearts and minds of software practitioners world-wide. Autonomous microservices, event-driven architecture, and CQRS are the dominant tools in the construction of contemporary business-centric applications. To top it off, the proliferation of device connectivity — IoT, mobile, wearables — is creating an upward pressure on the number of events a system must handle in near-real-time.

Let’s start by acknowledging that the term ‘fast’ is multi-faceted, complex, and highly ambiguous. Latency, throughput, jitter, are metrics that shape and influence one’s interpretation of the term. It is also inherently contextual: the industry and application domains in themselves set the norms and expectations around performance. Whether or not something is fast depends largely on one’s frame of reference.

Apache Kafka is optimized for throughput at the expense of latency and jitter, while preserving other desirable qualities, such as durability, strict record order, and at-least-once delivery semantics. When someone says ‘Kafka is fast’, and assuming they are at least mildly competent, you can assume they are referring to Kafka’s ability to safely accumulate and distribute a very high number of records in a short amount of time.

Historically, Kafka was born out of LinkedIn’s need to move a very large number of messages efficiently, amounting to multiple terabytes of data on an hourly basis. The individual message propagation delay was deemed of secondary importance, as was the variability of that time. After all, LinkedIn is not a financial institution that engages in high-frequency trading, nor is it an industrial control system that operates within deterministic deadlines. Kafka can be used to implement near-real-time (otherwise known as soft real-time) systems.

Note: For those unfamiliar with the term, ‘real-time’ does not mean ‘fast’, it means ‘predictable’. Specifically, real-time implies a hard upper bound, otherwise known as a deadline, on the time taken to complete an action. If the system as a whole is unable to meet this deadline each and every time, it cannot be classed as real-time. Systems that are able to perform within a probabilistic tolerance, are labelled as ‘near-real-time’. In terms of sheer throughput, real-time systems are often slower than their near-real-time or non-real-time counterparts.

There are two significant areas that Kafka draws upon for its speed, and they need to be discussed separately. The first relates to the low-level efficiency of the client and broker implementations. The second derives from the opportunistic parallelism of stream processing.

Broker performance

Log-structured persistence

Kafka utilizes a segmented, append-only log, largely limiting itself to sequential I/O for both reads and writes, which is fast across a wide variety of storage media. There is a wide misconception that disks are slow; however, the performance of storage media (particularly rotating media) is greatly dependent on access patterns. The performance of random I/O on a typical 7,200 RPM SATA disk is between three and four orders of magnitude slower when compared to sequential I/O. Furthermore, a modern operating system provides read-ahead and write-behind techniques that prefetch data in large block multiples and group smaller logical writes into large physical writes. Because of this, the difference between sequential I/O and random I/O is still evident in flash and other forms of solid-state non-volatile media, although it is far less dramatic compared to rotating media.

Record batching

Sequential I/O is blazingly fast on most media types, comparable to the peak performance of network I/O. In practice, this means that a well-designed log-structured persistence layer will keep up with the network traffic. In fact, quite often the bottleneck with Kafka isn’t the disk, but the network. So in addition to the low-level batching provided by the OS, Kafka clients and brokers will accumulate multiple records in a batch — for both reading and writing — before sending them over the network. Batching of records amortizes the overhead of the network round-trip, using larger packets and improving bandwidth efficiency.

Batch compression

The impact of batching is particularly obvious when compression is enabled, as compression becomes generally more effective as the data size increases. Especially when using text-based formats such as JSON, the effects of compression can be quite pronounced, with compression ratios typically ranging from 5x to 7x. Furthermore, record batching is largely done as a client-side operation, which transfers the load onto the client and has a positive effect not only on the network bandwidth but also on the brokers’ disk I/O utilization.

Cheap consumers

Unlike traditional MQ-style brokers which remove messages at point of consumption (incurring the penalty of random I/O), Kafka doesn’t remove messages after they are consumed — instead, it independently tracks offsets at each consumer group level. The progression of offsets themselves is published on an internal Kafka topic __consumer_offsets. Again, being an append-only operation, this is fast. The contents of this topic are further reduced in the background (using Kafka’s compaction feature) to only retain the last known offsets for any given consumer group.

Compare this model with more traditional message brokers which typically offer several contrasting message distribution topologies. On one hand is the message queue — a durable transport for point-to-point messaging, with no point-to-multipoint ability. On the other hand, a pub-sub topic allows for point-to-multipoint messaging but does so at the expense of durability. Implementing a durable point-to-multipoint messaging model in a traditional MQ requires maintaining a dedicated message queue for each stateful consumer. This creates both read and write amplification. On one hand, the publisher is forced to write to multiple queues. Alternatively, a fan-out relay may consume records from one queue and write to several others, but this only defers the point of amplification. On the other hand, several consumers are generating load on the broker — being a mixture of read and write I/O, both sequential and random.

Consumers in Kafka are ‘cheap’, insofar as they don’t mutate the log files (only the producer or internal Kafka processes are permitted to do that). This means that a large number of consumers may concurrently read from the same topic without overwhelming the cluster. There is still some cost in adding a consumer, but it is mostly sequential reads with a low rate of sequential writes. So it’s fairly normal to see a single topic being shared across a diverse consumer ecosystem.

Unflushed buffered writes

Another fundamental reason for Kafka’s performance, and one that is worth exploring further: Kafka doesn’t actually call fsync when writing to the disk before acknowledging the write; the only requirement for an ACK is that the record has been written to the I/O buffer. This is a little known fact, but a crucial one: in fact, this is what actually makes Kafka perform as if it were an in-memory queue — because for all intents and purposes Kafka is a disk-backed in-memory queue (limited by the size of the buffer/pagecache).

On the flip side, this form of writing is unsafe, as the failure of a replica can lead to a data loss even though the record has seemingly been acknowledged. In other words, unlike say a relational database, acknowledging a write alone does not imply durability. What makes Kafka durable is running several in-sync replicas; even if one were to fail, the others (assuming there is more than one) will remain operational — providing that the failure is uncorrelated (i.e. multiple replicas failing simultaneously due of a common upstream failure). So the combination of a non-blocking approach to I/O with no fsync, and redundant in-sync replicas give Kafka the combination of high throughput, durability, and availability.

Client-side optimisations

Most databases, queues, and other forms of persistent middleware are designed around the notion of an all-mighty server (or a cluster of servers), and fairly thin clients that communicate with the server(s) over a well-known wire protocol. Client implementations are generally considered to be significantly simpler than the server. As a result, the server will absorb the bulk of the load — the clients merely act as interfaces between the application code and the server.

Kafka takes a different approach to client design. A significant amount of work is performed on the client before records get to the server. This includes the staging of records in an accumulator, hashing the record keys to arrive at the correct partition index, checksumming the records and the compression of the record batch. The client is aware of the cluster metadata and periodically refreshes this metadata to keep abreast of any changes to the broker topology. This lets the client make low-level forwarding decisions; rather than sending a record blindly to the cluster and relying on the latter to forward it to the appropriate broker node, a producer client will forward writes directly to partition masters. Similarly, consumer clients are able to make intelligent decisions when sourcing records, potentially using replicas that geographically closer to the client when issuing read queries. (This feature is a more recent addition to Kafka, available as of version 2.4.0.)

Zero-copy

One of the typical sources of inefficiencies is copying byte data between buffers. Kafka uses a binary message format that is shared by the producer, the broker, and the consumer parties so that data chunks can flow end-to-end without modification, even if it’s compressed. While eliminating structural differences between communicating parties is an important step, it doesn’t in itself avoid the copying of data.

Kafka solves this problem on Linux and UNIX systems by using Java’s NIO framework, specifically, the transferTo() method of a java.nio.channels.FileChannel. This method permits the transfer of bytes from a source channel to a sink channel without involving the application as a transfer intermediary. To appreciate the difference that NIO makes, consider the traditional approach where a source channel is read into a byte buffer, then written to a sink channel as two separate operations:

File.read(fileDesc, buf, len);
Socket.send(socket, buf, len);

Diagrammatically, this can be represented using the following.

Although this looks simple enough, internally, the copy operation requires four context switches between user mode and kernel mode, and the data is copied four times before the operation is complete. The diagram below outlines the context switches at each step.

Looking at this in more detail —

  1. The initial read() causes a context switch from user mode to kernel mode. The file is read, and its contents are copied to a buffer in the kernel address space by the DMA (Direct Memory Access) engine. This is not the same buffer that was used in the code snippet.
  2. Prior to returning from read(), the kernel buffer is copied into the user-space buffer. At this point, our application can read the contents of the file.
  3. The subsequent send() will switch back into kernel mode, copying the user-space buffer into the kernel address space — this time into a different buffer associated with the destination socket. Behind the scenes, the DMA engine takes over, asynchronously copying the data from the kernel buffer to the protocol stack. The send() method does not wait for this prior to returning.
  4. The send() call returns, switching back to the user-space context.

In spite of its mode-switching inefficiencies and additional copying, the intermediate kernel buffer can actually improve performance in many cases. It can act as a read-ahead cache, asynchronously prefetching blocks, thereby front-running requests from the application. However, when the amount of requested data is significantly larger than the kernel buffer size, the kernel buffer becomes a performance bottleneck. Rather than copying the data directly, it forces the system to oscillate between user and kernel modes until all the data is transferred.

By contrast, the zero-copy approach is handled in a single operation. The snippet from the earlier example can be rewritten as a one-liner:

fileDesc.transferTo(offset, len, socket);

The zero-copy approach is illustrated below.

Under this model, the number of context switches is reduced to one. Specifically, the transferTo() method instructs the block device to read data into a read buffer by the DMA engine. This buffer is then copied another kernel buffer for staging to the socket. Finally, the socket buffer is copied to the NIC buffer by DMA.

As a result, we have reduced the number of copies from four to three, and only one of those copies involves the CPU. We have also reduced the number of context switches from four to two.

This is a massive improvement, but it’s not query zero-copy yet. The latter can be achieved as a further optimization when running Linux kernels 2.4 and later, and on network interface cards that support the gather operation. This is illustrated below.

Calling the transferTo() method causes the device to read data into a kernel read buffer by the DMA engine, as per the previous example. However, with the gather operation, there is no copying between the read buffer and the socket buffer. Instead, the NIC is given a pointer to the read buffer, along with the offset and the length, which is vacuumed up by DMA. At no point is the CPU involved in copying buffers.

Comparisons of traditional and zero-copy on file sizes ranging from a few megabytes to a gigabyte show performance gains by a factor of two to three in favor of zero-copy. But what’s more impressive, is that Kafka achieves this using a plain JVM with no native libraries or JNI code.

Avoiding the GC

The heavy use of channels, native buffers, and the page cache has one additional benefit — reducing the load on the garbage collector (GC). For example, running Kafka on a machine with 32 GB of RAM will result in 28–30 GB usable for the page cache, completely outside of the GC’s scope. The difference in throughput is minimal — in the region of several percentage points — as the throughput of a correctly-tuned GC can be quite high, especially when dealing with short-lived objects. The real gains are in the reduction of jitter; by avoiding the GC, the brokers are less likely to experience a pause that may impact the client, extending the end-to-end propagation delay of records.

To be fair, the avoidance of GC is less of a problem now, compared to what it used to be when Kafka was conceived. Modern GCs like Shenandoah and ZGC scale to huge, multi-terabyte heaps, and have tunable worst-case pause times, down to single-digit milliseconds. It is not uncommon these days to see JVM-based applications using large heap-based caches outperform off-heap designs.

Stream parallelism

The efficiency of log-structured I/O is one crucial aspect of performance, mostly affecting writes; Kafka’s treatment of parallelism in the topic structure and the consumer ecosystem is fundamental to its read performance. The combination produces an overall very high end-to-end messaging throughput. Concurrency is ingrained into its partitioning scheme and the operation of consumer groups, which is effectively a load-balancing mechanism within Kafka — distributing partition assignments approximately evenly among the individual consumer instances within the group. Compare this to a more traditional MQ: in an equivalent RabbitMQ setup, multiple concurrent consumers may read from a queue in a round-robin fashion, but in doing so they forfeit the notion of message ordering.

The partitioning mechanism also allows for the horizontal scalability of Kafka brokers. Every partition has a dedicated leader; any nontrivial topic (with multiple partitions) can, therefore, utilize the entire cluster of broker for writes. This is yet another point of distinction between Kafka and a message queue; where the latter utilizes clustering for availability, Kafka will genuinely balance the load across the brokers for availability, durability, and throughput.

The producer specifies the partition when publishing a record, assuming that you are publishing to a topic with multiple partitions. (One may have a single-partition topic, in which case this is a non-issue.) This may be accomplished either directly — by specifying a partition index, or indirectly — by way of a record key, which deterministically hashes to a consistent (i.e. same every time) partition index. Records sharing the same hash are guaranteed to occupy the same partition. Assuming a topic with multiple partitions, records with a different key will likely end up in different partitions. However, due to hash collisions, records with different hashes may also end up in the same partition. Such is the nature of hashing. If you understand how a hash table works, this is no different.

The actual processing of records is done by consumers, operating within an (optional) consumer group. Kafka guarantees that a partition may only be assigned to at most one consumer within its consumer group. (We say ‘at most’ to cover the case when all consumers are offline.) When the first consumer in a group subscribes to the topic, it will receive all partitions on that topic. When a second consumer subsequently joins, it will get approximately half of the partitions, relieving the first consumer of half of its prior load. This enables you to process an event stream in parallel, adding consumers as necessary (ideally, using an auto-scaling mechanism), providing that you have adequately partitioned your event stream.

Control of record throughput accomplished in two ways:

  1. The topic partitioning scheme. Topics should be partitioned to maximize the number of independent event sub-streams. In other words, record order should only be preserved where it is absolutely necessary. If any two records are not legitimately related in a causal sense, they shouldn’t be bound to the same partition. This implies the use of different keys, as Kafka will use a record’s key as a hashing source to derive its consistent partition mapping.
  2. The number of consumers in the group. You can increase the number of consumers to match the load of inbound records, up to the number of partitions in the topic. (You can have more consumers if you wish, but the partition count will place an upper bound on the number of active consumers which get at least one partition assignment; the remaining consumers will remain idle.) Note that a consumer could be a process or a thread. Depending on the type of workload that the consumer performs, you may be able to employ multiple individual consumer threads or process records in a thread pool.

If you were wondering whether Kafka is fast, how it achieves its renowned performance characteristics, or if it can scale to your use cases, you should hopefully by now have all the answers you need.

To make things abundantly clear, Kafka is not the fastest (that is, most throughput-capable) messaging middleware — there are other platforms capable of greater throughput — some are software-based and some are implemented in hardware. Nor is it the best throughput-latency compromise — Apache Pulsar is a promising technology that is scalable and achieves a better throughput-latency profile while offering identical ordering and durability guarantees. The rationale for adopting Kafka is that as a complete ecosystem, it remains unmatched overall. It exhibits excellent performance while offering an environment that is abundant and mature, but also involving — in spite of its size, Kafka is still growing at an enviable pace.

The designers and maintainers of Kafka have done an amazing job at devising a solution that is performance-oriented at its core. Few of its design elements feel like an afterthought or a bolt-on. From offloading of work to clients to the log-structured persistence on the broker, batching, compression, zero-copy I/O, and stream-level parallelism — Kafka throws down the gauntlet to just about any other message-oriented middleware, commercial or open-source. And most impressively, it does so without compromising on qualities such as durability, record order, and at-least-once delivery semantics.

Kafka is not the simplest of messaging platforms, and there is a fair bit to learn. One must come to grips with the concepts of a total and partial order, topics, partitions, consumers and consumer groups, before comfortably designing and building high-performance event-driven systems. And while the knowledge curve is substantial, the results will certainly be worth your while. If you are keen on taking the proverbial ‘red pill’, read the Introduction to Event Streaming with Kafka and Kafdrop.