Category: Java

Java is a class-based, object-oriented programming language that is designed to have as few implementation dependencies as possible.
Java applications are typically compiled to bytecode that can run on any Java virtual machine (JVM) regardless of the underlying computer architecture.

How to Schedule Tasks with Spring Boot
06
Feb
2021

How to Schedule Tasks with Spring Boot

In this article, You’ll learn how to schedule tasks in Spring Boot using @Scheduled annotation. You’ll also learn how to use a custom thread pool for executing all the scheduled tasks.

The @Scheduled annotation is added to a method along with some information about when to execute it, and Spring Boot takes care of the rest.

Spring Boot internally uses the TaskScheduler interface for scheduling the annotated methods for execution.

The purpose of this article is to build a simple project demonstrating all the concepts related to task scheduling.

Create the Project

Let’s use Spring Boot CLI to create the Project. Fire up your terminal and type the following command to generate the project –

$ spring init --name=scheduler-demo scheduler-demo 

Alternatively, You can generate the project using Spring Initializer web app. Just go to http://start.spring.io/, enter the Artifact’s value as “scheduler-demo” and click Generate to generate and download the project.

Once the project is generated, import it in your favorite IDE. The project’s directory structure should like this –

Spring Boot Scheduled Annotation Example Directory Structure

Enable Scheduling

You can enable scheduling simply by adding the @EnableScheduling annotation to the main application class or one of the Configuration classes.

Open SchedulerDemoApplication.java and add @EnableScheduling annotation like so –

package com.example.schedulerdemo;

import org.springframework.boot.SpringApplication;
import org.springframework.boot.autoconfigure.SpringBootApplication;
import org.springframework.scheduling.annotation.EnableScheduling;

@SpringBootApplication
@EnableScheduling
public class SchedulerDemoApplication {

	public static void main(String[] args) {
		SpringApplication.run(SchedulerDemoApplication.class, args);
	}
}

Scheduling Tasks

Scheduling a task with Spring Boot is as simple as annotating a method with @Scheduled annotation, and providing few parameters that will be used to decide the time at which the task will run.

Before adding tasks, Let’s first create the container for all the scheduled tasks. Create a new class called ScheduledTasks inside com.example.schedulerdemo package with the following contents –

package com.example.schedulerdemo;

import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.springframework.scheduling.annotation.Scheduled;
import org.springframework.stereotype.Component;

import java.time.LocalDateTime;
import java.time.format.DateTimeFormatter;
import java.util.concurrent.TimeUnit;

@Component
public class ScheduledTasks {
    private static final Logger logger = LoggerFactory.getLogger(ScheduledTasks.class);
    private static final DateTimeFormatter dateTimeFormatter = DateTimeFormatter.ofPattern("HH:mm:ss");

    public void scheduleTaskWithFixedRate() {}

    public void scheduleTaskWithFixedDelay() {}

    public void scheduleTaskWithInitialDelay() {}

    public void scheduleTaskWithCronExpression() {}
}

The class contains four empty methods. We’ll look at the implementation of all the methods one by one.

All the scheduled methods should follow the following two criteria –

  • The method should have a void return type.
  • The method should not accept any arguments.

Cool! Let’s now jump into the implementation.

1. Scheduling a Task with Fixed Rate

You can schedule a method to be executed at a fixed interval by using fixedRate parameter in the @Scheduled annotation. In the following example, The annotated method will be executed every 2 seconds.

@Scheduled(fixedRate = 2000)
public void scheduleTaskWithFixedRate() {
    logger.info("Fixed Rate Task :: Execution Time - {}", dateTimeFormatter.format(LocalDateTime.now()) );
}
# Sample Output
Fixed Rate Task :: Execution Time - 10:26:58
Fixed Rate Task :: Execution Time - 10:27:00
Fixed Rate Task :: Execution Time - 10:27:02
....
....

The fixedRate task is invoked at the specified interval even if the previous invocation of the task is not finished.

2. Scheduling a Task with Fixed Delay

You can execute a task with a fixed delay between the completion of the last invocation and the start of the next, using fixedDelay parameter.

The fixedDelay parameter counts the delay after the completion of the last invocation.

Consider the following example –

@Scheduled(fixedDelay = 2000)
public void scheduleTaskWithFixedDelay() {
    logger.info("Fixed Delay Task :: Execution Time - {}", dateTimeFormatter.format(LocalDateTime.now()));
    try {
        TimeUnit.SECONDS.sleep(5);
    } catch (InterruptedException ex) {
        logger.error("Ran into an error {}", ex);
        throw new IllegalStateException(ex);
    }
}

Since the task itself takes 5 seconds to complete and we have specified a delay of 2 seconds between the completion of the last invocation and the start of the next, there will be a delay of 7 seconds between each invocation –

# Sample Output
Fixed Delay Task :: Execution Time - 10:30:01
Fixed Delay Task :: Execution Time - 10:30:08
Fixed Delay Task :: Execution Time - 10:30:15
....
....

3. Scheduling a Task With Fixed Rate and Initial Delay

You can use initialDelay parameter with fixedRate and fixedDelay to delay the first execution of the task with the specified number of milliseconds.

In the following example, the first execution of the task will be delayed by 5 seconds and then it will be executed normally at a fixed interval of 2 seconds –

@Scheduled(fixedRate = 2000, initialDelay = 5000)
public void scheduleTaskWithInitialDelay() {
    logger.info("Fixed Rate Task with Initial Delay :: Execution Time - {}", dateTimeFormatter.format(LocalDateTime.now()));
}
# Sample output (Server Started at 10:48:46)
Fixed Rate Task with Initial Delay :: Execution Time - 10:48:51
Fixed Rate Task with Initial Delay :: Execution Time - 10:48:53
Fixed Rate Task with Initial Delay :: Execution Time - 10:48:55
....
....

4. Scheduling a Task using Cron Expression

If the above simple parameters can not fulfill your needs, then you can use cron expressions to schedule the execution of your tasks.

In the following example, I have scheduled the task to be executed every minute –

@Scheduled(cron = "0 * * * * ?")
public void scheduleTaskWithCronExpression() {
    logger.info("Cron Task :: Execution Time - {}", dateTimeFormatter.format(LocalDateTime.now()));
}
# Sample Output
Cron Task :: Execution Time - 11:03:00
Cron Task :: Execution Time - 11:04:00
Cron Task :: Execution Time - 11:05:00

Running @Scheduled Tasks in a Custom Thread Pool

By default, all the @Scheduled tasks are executed in a default thread pool of size one created by Spring.

You can verify that by logging the name of the current thread in all the methods –

logger.info("Current Thread : {}", Thread.currentThread().getName());

All the methods will print the following –

Current Thread : pool-1-thread-1

But hey, You can create your own thread pool and configure Spring to use that thread pool for executing all the scheduled tasks.

Create a new package config inside com.example.schedulerdemo, and then create a new class called SchedulerConfig inside config package with the following contents –

package com.example.schedulerdemo.config;

import org.springframework.context.annotation.Configuration;
import org.springframework.scheduling.annotation.SchedulingConfigurer;
import org.springframework.scheduling.concurrent.ThreadPoolTaskScheduler;
import org.springframework.scheduling.config.ScheduledTaskRegistrar;

@Configuration
public class SchedulerConfig implements SchedulingConfigurer {
    private final int POOL_SIZE = 10;

    @Override
    public void configureTasks(ScheduledTaskRegistrar scheduledTaskRegistrar) {
        ThreadPoolTaskScheduler threadPoolTaskScheduler = new ThreadPoolTaskScheduler();

        threadPoolTaskScheduler.setPoolSize(POOL_SIZE);
        threadPoolTaskScheduler.setThreadNamePrefix("my-scheduled-task-pool-");
        threadPoolTaskScheduler.initialize();

        scheduledTaskRegistrar.setTaskScheduler(threadPoolTaskScheduler);
    }
}

That’s all you need to do for configuring Spring to use your own thread pool instead of the default one.

If you log the name of the current thread in the scheduled methods now, you’ll get the output like so –

Current Thread : my-scheduled-task-pool-1
Current Thread : my-scheduled-task-pool-2

# etc...

Conclusion

In this article, you learned how to schedule tasks in Spring Boot using @Scheduled annotation. You also learned how to use a custom thread pool for running these tasks.

The Scheduling abstraction provided by Spring Boot works pretty well for simple use-cases. But if you have more advanced use cases like Persistent JobsClusteringDynamically adding and triggering new jobs then check out the following article –

Spring Boot Quartz Scheduler Example: Building an Email Scheduling App

Thank you for reading. See you in the next Post!

JPA / Hibernate Composite Primary Key Example with Spring Boot
03
Mar
2021

JPA / Hibernate Composite Primary Key Example with Spring Boot

In this article, You’ll learn how to map a composite primary key in Hibernate using JPA’s @Embeddable and @EmbeddedId annotations.

Let’s say that We have an application that manages Employees of various companies. Every employee has a unique employeeId within his company. But the same employeeId can be present in other companies as well, So we can not uniquely identity an employee just by his employeeId.

To identify an employee uniquely, we need to know his employeeId and companyId both. Check out the following Employees table that contains a composite primary key which includes both the employeeId and companyId columns –

Hibernate Composite Primary Key Example Table Structure

Let’s create a project from scratch and learn how to map such composite primary key using JPA and Hibernate.

Creating the Project

You can generate the project quickly using Spring Boot CLI by typing the following command in the terminal –

spring init -n=jpa-composite-primary-key-demo -d=web,jpa,mysql --package-name=com.example.jpa jpa-composite-primary-key-demo

Alternatively, You can also use Spring Initializr web app to generate the project. Follow the instructions below to generate the app using Spring Initializr web app –

  1. Open http://start.spring.io
  2. Enter Artifact as “jpa-composite-primary-key-demo”
  3. Click Options dropdown to see all the options related to project metadata.
  4. Change Package Name to “com.example.jpa”
  5. Select WebJPA and Mysql dependencies.
  6. Click Generate to generate and download the project.

Following is the directory structure of the complete application for your reference –

JPA, Hibernate, Spring Boot Composite Primary Key Example Directory Structure

(Your bootstrapped project won’t have model and repository packages and all the other classes. We’ll create them as we proceed to next sections)

Configuring the Database and Hibernate Log Levels

Let’s add the MySQL database URL, username and password configurations in src/main/resources/application.properties file –

# DATASOURCE (DataSourceAutoConfiguration & DataSourceProperties)
spring.datasource.url=jdbc:mysql://localhost:3306/jpa_composite_pk_demo?useSSL=false&serverTimezone=UTC&useLegacyDatetimeCode=false
spring.datasource.username=root
spring.datasource.password=root

# Hibernate

# The SQL dialect makes Hibernate generate better SQL for the chosen database
spring.jpa.properties.hibernate.dialect = org.hibernate.dialect.MySQL5InnoDBDialect

# Hibernate ddl auto (create, create-drop, validate, update)
spring.jpa.hibernate.ddl-auto = update

logging.level.org.hibernate.SQL=DEBUG
logging.level.org.hibernate.type=TRACE

Apart from MySQL database configurations, I’ve also specified hibernate log levels and other properties.

The property spring.jpa.hibernate.ddl-auto = update keeps the Entity types in your application and the mapped database tables in sync. Whenever you update a domain entity, the corresponding mapped table in the database will also get updated when you restart the application next time.

This is great for development because you don’t need to manually create or update the tables. They will automatically be created/updated based on the Entity classes in your application.

Before proceeding to the next section, Please make sure that you create a MySQL database named jpa_composite_pk_demo and change spring.datasource.username and spring.datasource.password properties as per your MySQL installation.

Defining the Domain model

A composite primary key is mapped using an Embeddable type in hibernate. We’ll first create an Embeddable type called EmployeeIdentity containing the employeeId and companyId fields, and then create the Employee entity which will embed the EmployeeIdentity type.

Create a new package named model inside com.example.jpa package and then add the following classes inside the model package –

1. EmployeeIdentity – Embeddable Type

package com.example.jpa.model;

import javax.persistence.Embeddable;
import javax.validation.constraints.NotNull;
import javax.validation.constraints.Size;
import java.io.Serializable;

@Embeddable
public class EmployeeIdentity implements Serializable {
    @NotNull
    @Size(max = 20)
    private String employeeId;

    @NotNull
    @Size(max = 20)
    private String companyId;

    public EmployeeIdentity() {

    }

    public EmployeeIdentity(String employeeId, String companyId) {
        this.employeeId = employeeId;
        this.companyId = companyId;
    }

    public String getEmployeeId() {
        return employeeId;
    }

    public void setEmployeeId(String employeeId) {
        this.employeeId = employeeId;
    }

    public String getCompanyId() {
        return companyId;
    }

    public void setCompanyId(String companyId) {
        this.companyId = companyId;
    }

    @Override
    public boolean equals(Object o) {
        if (this == o) return true;
        if (o == null || getClass() != o.getClass()) return false;

        EmployeeIdentity that = (EmployeeIdentity) o;

        if (!employeeId.equals(that.employeeId)) return false;
        return companyId.equals(that.companyId);
    }

    @Override
    public int hashCode() {
        int result = employeeId.hashCode();
        result = 31 * result + companyId.hashCode();
        return result;
    }
}

2. Employee – Domain model

package com.example.jpa.model;

import org.hibernate.annotations.NaturalId;
import javax.persistence.Column;
import javax.persistence.EmbeddedId;
import javax.persistence.Entity;
import javax.persistence.Table;
import javax.validation.constraints.NotNull;
import javax.validation.constraints.Email;
import javax.validation.constraints.Size;

@Entity
@Table(name = "employees")
public class Employee {

    @EmbeddedId
    private EmployeeIdentity employeeIdentity;

    @NotNull
    @Size(max = 60)
    private String name;

    @NaturalId
    @NotNull
    @Email
    @Size(max = 60)
    private String email;

    @Size(max = 15)
    @Column(name = "phone_number", unique = true)
    private String phoneNumber;

    public Employee() {

    }

    public Employee(EmployeeIdentity employeeIdentity, String name, String email, String phoneNumber) {
        this.employeeIdentity = employeeIdentity;
        this.name = name;
        this.email = email;
        this.phoneNumber = phoneNumber;
    }

    // Getters and Setters (Omitted for brevity)
}

In the Employee class, We use @EmbeddedId annotation to embed the EmployeeIdentity type and mark it as a primary key.

Creating the Repository

Next, Let’s create the repository for accessing the Employee data from the database. First, Create a new package named repository inside com.example.jpa package, then add the following interface inside the repository package –

package com.example.jpa.repository;

import com.example.jpa.model.Employee;
import com.example.jpa.model.EmployeeIdentity;
import org.springframework.data.jpa.repository.JpaRepository;
import org.springframework.stereotype.Repository;

@Repository
public interface EmployeeRepository extends JpaRepository<Employee, EmployeeIdentity> {

}

Code to test the Composite Primary Key Mapping

Finally, Let’s write some code to test the composite primary key mapping. Open the main class JpaCompositePrimaryKeyDemoApplication.java and replace it with the following code –

package com.example.jpa;

import com.example.jpa.model.Employee;
import com.example.jpa.model.EmployeeIdentity;
import com.example.jpa.repository.EmployeeRepository;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.boot.CommandLineRunner;
import org.springframework.boot.SpringApplication;
import org.springframework.boot.autoconfigure.SpringBootApplication;

@SpringBootApplication
public class JpaCompositePrimaryKeyDemoApplication implements CommandLineRunner {

    @Autowired
    private EmployeeRepository employeeRepository;

    public static void main(String[] args) {
        SpringApplication.run(JpaCompositePrimaryKeyDemoApplication.class, args);
    }

    @Override
    public void run(String... args) throws Exception {
        // Cleanup employees table
        employeeRepository.deleteAllInBatch();

        // Insert a new Employee in the database
        Employee employee = new Employee(new EmployeeIdentity("E-123", "D-457"),
                "Yaniv Levy",
                "yaniv@fusebes.com",
                "+91-9999999999");

        employeeRepository.save(employee);
    }
}

We first clean up the Employee table and then insert a new Employee record with an employeeId and a companyId to test the setup.

You can run the application by typing mvn spring-boot:run from the root directory of the project. The Employee record will be inserted in the database once the application is successfully started.

Querying using the Composite Primary Key

Let’s now see some query examples using the composite primary key –

1. Retrieving an Employee using the composite primary key – (employeeId and companyId)

// Retrieving an Employee Record with the composite primary key
employeeRepository.findById(new EmployeeIdentity("E-123", "D-457"));

2. Retrieving all employees of a particular company

Let’s say that you want to retrieve all the employees of a company by companyId. For doing this, just add the following method in the EmployeeRepository interface.

@Repository
public interface EmployeeRepository extends JpaRepository<Employee, EmployeeIdentity> {
    /* 
       Spring Data JPA will automatically parse this method name 
       and create a query from it
    */
    List<Employee> findByEmployeeIdentityCompanyId(String companyId);
}

That’s all! You don’t need to implement anything. Spring Data JPA will dynamically generate a query using the method name. You can use the above method in the main class to retrieve all the employees of a company like this –

// Retrieving all the employees of a particular company
employeeRepository.findByEmployeeIdentityCompanyId("D-457");

Conclusion

Congratulations guys! In this article, you learned how to implement a composite primary key in hibernate using @Embeddable and @EmbeddedId annotations.

Thanks for reading. See you in the next post.

Maximum Contiguous Subarray Sum
06
Feb
2021

Maximum Contiguous Subarray Sum

Maximum Contiguous Subarray Sum problem statement

Given an array of integers, find the contiguous subarray (containing at least one number) which has the largest sum and return its sum.

Example:

Input: [-2,1,-3,4,-1,2,1,-5,4],
Output: 6
Explanation: The subarray [4,-1,2,1] has the largest sum = 6.

Maximum Contiguous Subarray Sum solution in Java

This problem can be solved using Kadane’s algorithm in O(n) time complexity.

The algorithm iterates over all the elements of the array (nums) and computes the maximum sum ending at every index (maxEndingHere). Here is how maxEndingHere is calculated:

  • Initialize maxEndingHere to nums[0]
  • Iterate through the array (1..n). For every iteration:
    • If maxEndingHere + nums[i] < nums[i], that means the previous max was negative and therefore we reset maxEndingHere to nums[i].
    • Otherwise, we set maxEndingHere = maxEndingHere + nums[i]
  • We also keep a variable maxSoFar which indicates the maximum sum found so far. And, with every iteration, we check if the current max (maxEndingHere) is greater than maxSoFar. If yes, then we update the value of maxSoFar.

Here is the complete solution:

import java.util.Scanner;

class MaximumSubarray {

  private static int findMaxSubarraySum(int[] nums) {
    int n = nums.length;
    int maxSoFar = nums[0];
    int maxEndingHere = nums[0];

    for(int i = 1; i < n; i++) {
      if(maxEndingHere + nums[i] < nums[i]) {
        maxEndingHere = nums[i];
      } else {
        maxEndingHere = maxEndingHere + nums[i];
      }

      if(maxEndingHere > maxSoFar) {
        maxSoFar = maxEndingHere;
      }
    }
    return maxSoFar;
  }

  public static void main(String[] args) {
    Scanner keyboard = new Scanner(System.in);
    int n = keyboard.nextInt();
    int[] nums = new int[n];
    for(int i = 0; i < n; i++) {
      nums[i] = keyboard.nextInt();
    }

    System.out.println(findMaxSubarraySum(nums));
  }
}
# Output
$ javac MaximumSubarray.java
$ java MaximumSubarray
9 -2 1 -3 4 -1 2 1 -5 4
6

Maximum Contiguous Subarray with Indices

It’s also possible to find out the start and end indices of the maximum contiguous subarray. Here is a modification of the above program which does that and prints the subarray –

import java.util.Scanner;

class MaximumSubarray {

  private static int[] findMaxSubarrayIndices(int[] nums) {
    int n = nums.length;
    int maxSoFar = nums[0];
    int maxEndingHere = nums[0];
    int currentStart = 0, maxStart = 0, maxEnd = 0;

    for(int i = 1; i < n; i++) {
      if(maxEndingHere + nums[i] < nums[i]) {
        maxEndingHere = nums[i];
        currentStart = i;
      } else {
        maxEndingHere = maxEndingHere + nums[i];
      }

      if(maxEndingHere > maxSoFar) {
        maxSoFar = maxEndingHere;
        maxStart = currentStart;
        maxEnd = i;
      }
    }

    return new int[]{maxStart, maxEnd};
  }

  public static void main(String[] args) {
    Scanner keyboard = new Scanner(System.in);
    int n = keyboard.nextInt();
    int[] nums = new int[n];
    for(int i = 0; i < n; i++) {
      nums[i] = keyboard.nextInt();
    }

    int[] indices = findMaxSubarrayIndices(nums);
    int sum = 0;
    System.out.print("Max contiguous subarray: ");
    for(int i = indices[0]; i <= indices[1]; i++) {
      System.out.print(nums[i] + " ");
      sum += nums[i];
    }
    System.out.println();
    System.out.println("Max contiguous subarray sum: " + sum);
  }
}
# Output
$ javac MaximumSubarray.java
$ java MaximumSubarray
9 -2 1 -3 4 -1 2 1 -5 4
Max contiguous subarray: 4 -1 2 1
Max contiguous subarray sum: 6
Build A Web Crawler with Java
02
Jan
2022

Build A Web Crawler with Java

Creating a web crawler is a smart way of retrieving useful information available online. With a web crawler, you can scan the Internet, browse through individual websites, and analyze and extract their content.

The Java programming language provides a simple way of building a web crawler and harvesting data from websites. You can use the extracted data for various use cases, such as for analytical purposes, providing a service that uses third-party data, or generating statistical data.

In this article, we’ll walk you through the process of building a web crawler using Java and ProxyCrawl.

What you’ll need

Typically, crawling web data involves creating a script that sends a request to the targeted web page, accesses its underlying HTML code, and scrapes the required information.

To accomplish that objective, you’ll need the following:

Before we develop the crawling logic, let’s clear the air why using ProxyCrawl is important for web crawling.

Why use ProxyCrawl for Crawling

ProxyCrawl is a powerful data crawling and scraping tool you can use to harvest information from websites fast and easily.

Here are some reasons why you should use it for crawling online data:

  • Easy to use It comes with a simple API that you can set up quickly without any programming hurdles. With just a few lines of code, you can start using the API to crawl websites and retrieve their content.
  • Supports advanced crawling ProxyCrawl allows you to perform advanced web crawling and scrape data from complicated websites. Since it supports JavaScript rendering, ProxyCrawl lets you extract data from dynamic websites. It offers a headless browser that allows you to extract what real users see on their web browsers—even if a site is created using modern frameworks like Angular or React.js.
  • Bypass crawling obstacles ProxyCrawl can handle all the restrictions often associated with crawling online data. It has an extensive network of proxies as well as more than 17 data centers around the world. You can use it to avoid access restrictions, resolve CAPTCHAs, and evade other anti-scraping measures implemented by web applications. What’s more, you can crawl websites while remaining anonymous; you’ll not worry about exposing your identity.
  • Free trial account You can test how ProxyCrawl works without giving out your payment details. The free account comes with 1,000 credits for trying out the tool’s capabilities.

How ProxyCrawl Works

ProxyCrawl provides the Crawling API for crawling and scraping data from websites. You can easily integrate the API in your Java development project and retrieve information from web pages smoothly.

Each request made to the Crawling API starts with the following base part:

https://api.proxycrawl.com

Also, you’ll need to add the following mandatory parameters to the API:

  • Authentication token
  • URL

The authentication token is a unique token that authorizes you to use the Crawling API. Once you sign up for an account, ProxyCrawl will give you two types of tokens:

  • Normal token This is for making generic crawling requests.
  • JavaScript token This is for crawling dynamic websites. It provides you with headless browser capabilities for crawling web pages rendered using JavaScript. As pointed out earlier, it’s a useful way of crawling advanced websites.

Here is how to add the authentication token to your API request:

https://api.proxycrawl.com/?token=INSERT_TOKEN

The second mandatory parameter is the URL to crawl. It should start with HTTP or HTTPS, and be completely encoded. Encoding converts the URL string into a format that can be transferred over the Internet validly and easily.

Here is how to insert the URL to your API request:

https://api.proxycrawl.com/?token=INSERT_TOKEN&url=INSERT_URL

If you run the above line—for example, on your terminal using cURL or pasting it on a browser’s address bar—it’ll execute the API request and return the entire HTML source code of the targeted web page.

It’s that easy and simple!

If you want to perform advanced crawling, you may add other parameters to the API request. For example, when using the JavaScript token, you can add the page_wait parameter to instruct the browser to wait for the specified number of milliseconds before the resulting HTML code is captured.

Here is an example:

https://api.proxycrawl.com/?token=INSERT_TOKEN&page_wait=1000&url=INSERT_URL

Building a Web Crawler in Java and ProxyCrawl

In this Java web crawling tutorial, we’ll use the HttpClient API to create the crawling logic. The API was introduced in Java 11, and it comes with lots of useful features for sending requests and retrieving their responses.

The HttpClient API supports both HTTP/1.1 and HTTP/2. By default, it uses the HTTP/2 protocol to send requests. If a request is sent to a server that does not already support HTTP/2, it will automatically be downgraded to HTTP/1.

Furthermore, its requests can be sent asynchronously or synchronously, it handles requests and response bodies as reactive-streams, and uses the common builder pattern.

The API is comprised of three core classes:

  • HttpRequest
  • HttpClient
  • HttpResponse

Let’s talk about each of them in more detail.

1. HttpRequest

The HttpRequest, as the name implies, is an object encapsulating the HTTP request to be sent. To create new instances of HttpRequest, call HttpRequest.newBuilder(). After it has been created, the request is immutable and can be sent multiple times.

The Builder class comes with different methods for configuring the request.

These are the most common methods:

  • URI method
  • Request method
  • Protocol version method
  • Timeout method

Let’s talk about each of them in more detail.

a) URI method

The first thing to do when configuring the request is to set the URL to crawl. We can do so by calling the uri() method on the Builder instance. We’ll also use the URI.create() method to create the URI by parsing the string of the URL we intend to crawl.

Here is the code:

String url =
URLEncoder.encode(“https://www.forextradingbig.com/7-reasons-why-you-should
-quit-forex-trading/”, StandardCharsets.UTF_8.name());

HttpRequest request = HttpRequest.newBuilder()
.uri(URI.create(“https://api.proxycrawl.com/?token=INSERT_TOKEN&url=”
+ url))

Notice that we provided the URL string using ProxyCrawl’s settings. This is the web page we intend to scrape its contents.

We also encoded the URL using the Java URLEncoder class. As earlier mentioned, ProxyCrawl requires URLs to be encoded.

b) Request method

The next thing to do is to specify the HTTP method to be used for making the request. We can call any of the following methods from Builder:

  • GET()
  • POST()
  • PUT()
  • DELETE()

In this case, since we want to request data from the target web page, we’ll use the GET() method.

Here is the code:

HttpRequest request = HttpRequest.newBuilder()
.GET()

So far, HttpRequest has all the parameters that should be passed to HttpClient. However, you may need to include other parameters, such as the HTTP protocol version and timeout.

Let’s see how you can add the additional parameters.

c) Protocol version method

As earlier mentioned, the HttpClient API uses the HTTP/2 protocol by default. Nonetheless, you can specify the version of the HTTP protocol you want to use.

Here is the code:

HttpRequest request = HttpRequest.newBuilder()
.version(HttpClient.Version.HTTP_2)

d) Timeout method

You can set the amount of time to wait before a response is received. Once the defined period expires, an HttpTimeoutException will be thrown. By default, the timeout is set to infinity.

You can define timeout by calling the timeout() method on the builder instance. You’ll also need to pass the Duration object to specify the amount of time to wait.

Here is the code:

HttpRequest request = HttpRequest.newBuilder()
.timeout(Duration.ofSeconds(20))

2. HttpClient

The HttpClient class is the main entry point of the API—it acts as a container for the configuration details shared among multiple requests. It is the HTTP client used for sending requests and receiving responses.

You can call either the HttpClient.newBuilder() or the HttpClient.newHttpClient() method to instantiate it. After an instance of the HttpClient has been created, it’s immutable.

The HttpClient class offers several helpful and self-describing methods you can use when working with requests and responses.

These are some things you can do:

  • Set protocol version
  • Set redirect policy
  • Send synchronous and asynchronous requests

Let’s talk about each of them in more detail.

a) Set protocol version

As earlier mentioned, the HttpClient class uses the HTTP/2 protocol by default. However, you can set your preferred protocol version, either HTTP/1.1 or HTTP/2.

Here is an example:

HttpClient client = HttpClient.newBuilder()
.version(Version.HTTP_1_1)

b) Set redirect policy

If the targeted web page has moved to a different address, you’ll get a 3xx HTTP status code. Since the address of the new URI is usually provided with the status code information, setting the correct redirect policy can make HttpClient forward the request automatically to the new location.

You can set it by using the followRedirects() method on the Builder instance.

Here is an example:

HttpClient client = HttpClient.newBuilder()
.followRedirects(Redirect.NORMAL)

c) Send synchronous and asynchronous requests

HttpClient supports two ways of sending requests:

  • Synchronously by using the send() method. This blocks the client until the response is received, before continuing with the rest of the execution.

Here is an example:

HttpResponse<String> response = client.send(request,
BodyHandlers.ofString());

Note that we used BodyHandlers and called the ofString() method to return the HTML response as a string.

  • Asynchronously by using the sendAsync() method. This does not wait for the response to be received; it’s non-blocking. Once the sendAsync() method is called, it returns instantly with a CompletableFuture< HttpResponse >, which finalizes once the response is received. The returned CompletableFuture can be joined using various techniques to define dependencies among varied asynchronous tasks.

Here is an example:

CompletableFuture<HttpResponse<String>> response = HttpClient.newBuilder()
.sendAsync(request, HttpResponse.BodyHandler.ofString());

3. HttpResponse

The HttpResponse, as the name implies, represents the response received after sending an HttpRequest. HttpResponse offers different helpful methods for handling the received response.

These are the most important methods:

  • statusCode() This method returns the status code of the response. It’s of int type
  • Body() This method returns a body for the response. The return type is based on the kind of response BodyHandler parameter that is passed to the send() method.

Here is an example:

// Handling the response body as a String
HttpResponse<String> response = client
.send(request, BodyHandlers.ofString());

// Printing response body
System.out.println(response.body());

// Printing status code
System.out.println(response.statusCode());
// Handling the response body as a file
HttpResponse<Path> response = client
.send(request, BodyHandlers.ofFile(Paths.get(“myexample.html”)));

Synchronous Example

Here is an example that uses the HttpClient synchronous method to crawl a web page and output its content:

package javaHttpClient;

import java.io.IOException;
import java.net.URI;
import java.net.URLEncoder;
import java.net.http.HttpClient;
import java.net.http.HttpRequest;
import java.net.http.HttpResponse;
import java.net.http.HttpResponse.BodyHandlers;
import java.nio.charset.StandardCharsets;

public class SyncExample {

public static void main(String[] args) throws IOException, InterruptedException {

// Encoding the URL
String url = URLEncoder.encode(“https://www.forextradingbig.com/7-reasons-why-you-should-quit-forex-trading/”, StandardCharsets.UTF_8.name());

// Instantiating HttpClient
HttpClient client = HttpClient.newHttpClient();

// Configuring HttpRequest
HttpRequest request = HttpRequest.newBuilder()
.GET()
.uri(URI.create(“https://api.proxycrawl.com/?token=INSERT_TOKEN&url=” + url))
.build();

// Handling the response
HttpResponse<String> response = client.send(request, BodyHandlers.ofString());
System.out.println(response.body());
}

}

Here is the output:

Asynchronous Example

When using the HttpClient asynchronous method to crawl a web page, the sendAsync() method is called, instead of send().

Here is an example:

package javaHttpClient;

import java.io.IOException;
import java.net.URI;
import java.net.URLEncoder;
import java.net.http.HttpClient;
import java.net.http.HttpRequest;
import java.net.http.HttpResponse;
import java.nio.charset.StandardCharsets;
import java.util.concurrent.CompletableFuture;
import java.util.concurrent.ExecutionException;
import java.util.concurrent.TimeUnit;
import java.util.concurrent.TimeoutException;

public class AsyncExample {


public static void main(String[] args) throws IOException, InterruptedException, ExecutionException, TimeoutException {

// Encoding the URL
String url = URLEncoder.encode(“https://www.forextradingbig.com/7-reasons-why-you-should-quit-forex-trading/”, StandardCharsets.UTF_8.name());

// Instantiating HttpClient
HttpClient client = HttpClient.newHttpClient();

// Configuring HttpRequest
HttpRequest request = HttpRequest.newBuilder()
.GET()
.version(HttpClient.Version.HTTP_2)
.uri(URI.create(“https://api.proxycrawl.com/?token=INSERT_TOKEN&url=” + url))
.build();

// Handling the response
CompletableFuture<HttpResponse<String>> response =
client.sendAsync(request, HttpResponse.BodyHandlers.ofString());

String result = response.thenApply(HttpResponse::body).get(5, TimeUnit.SECONDS);

System.out.println(result);

}

}

Conclusion

That’s how to build a web crawler in Java. The HttpClient API, which was introduced in Java 11, makes it easy to send and handle responses from a server.

And if the API is combined with a versatile tool like ProxyCrawl, it can make web crawling tasks smooth and rewarding.

With ProxyCrawl, you can create a scraper that can help you to retrieve information from websites anonymously and without worrying about being blocked.

It’s the tool you need to take your crawling efforts to the next level.

Click here to create a free ProxyCrawl account.

Happy scraping!

Spring Boot @Configuration Properties_ Binding external configurations to POJO classes
06
Feb
2021

Spring Boot @ConfigurationProperties: Binding external configurations to POJO classes

External configurations allow you to work with the same code in different environments. They also provide you the flexibility to tune your application from a single place.

In this article, you’ll learn how to define and use external configurations in Spring Boot with a very simple annotation based API called @ConfigurationProperties.

@ConfigurationProperties bind external configurations to a strongly typed bean in your application code. You can inject and use this bean throughout your application code just like any other spring bean.

Let the exploring begin!

Creating the Application

Let’s create a sample application to learn how to define and use external configurations in spring boot.

We’ll use Spring Boot CLI to initialize the project. Fire up your terminal and type the following command to generate the project –

spring init --dependencies=web --name=config-properties-demo --package-name=com.example.demo config-properties-demo

Alternatively, You may also bootstrap the application from Spring Initializr website by following the instructions below –

  1. Go to http://start.spring.io
  2. Enter config-properties-demo in the Artifact field.
  3. Add Web in the dependencies section.
  4. Click Generate Project to download the project.

Following is the directory structure of the complete application for your reference-

Spring Boot Environment Based Configuration Properties Example

Defining Properties

Spring Boot application loads configuration properties from application.properties file located in the classpath by default.

Open src/main/resources/application.properties file and add the following properties to it

## Top level app properties
app.name=ConfigurationPropertiesDemoApp
app.description=${app.name} is a spring boot app that demonstrates how to use external configuration properties
app.upload-dir=/uploads

app.connect-timeout=500ms
app.read-timeout=10s

## Nested Object Properties (security)
app.security.username=user
app.security.password=123456
app.security.roles=USER,ADMIN,PARTNER   # List Property
app.security.enabled=true

## Map Properties (permissions)
app.security.permissions.CAN_VIEW_POSTS=true
app.security.permissions.CAN_EDIT_POSTS=true
app.security.permissions.CAN_DELETE_POSTS=false
app.security.permissions.CAN_VIEW_USERS=true
app.security.permissions.CAN_EDIT_USERS=true
app.security.permissions.CAN_DELETE_USERS=false

Binding external properties to a POJO class using @ConfigurationProperties

Let’s now see how we can bind the properties defined in the application.properties file to a POJO class using @ConfigurationProperties.

The @ConfigurationProperties annotation takes a prefix parameter, and binds all the properties with the specified prefix to the POJO class –

package com.example.demo.config;

import org.springframework.boot.context.properties.ConfigurationProperties;
import org.springframework.boot.convert.DurationUnit;

import java.time.Duration;
import java.time.temporal.ChronoUnit;
import java.util.ArrayList;
import java.util.HashMap;
import java.util.List;
import java.util.Map;

@ConfigurationProperties(prefix = "app")
public class AppProperties {
    private String name;
    private String description;
    private String uploadDir;
    private Duration connectTimeout = Duration.ofMillis(1000);
    @DurationUnit(ChronoUnit.SECONDS)
    private Duration readTimeout = Duration.ofSeconds(30);
    private final Security security = new Security();

    // Getters and Setters (Omitted for brevity)

    public static class Security {
        private String username;
        private String password;
        private List<String> roles = new ArrayList<>();
        private boolean enabled;
        private Map<String, String> permissions = new HashMap<>();

        // Getters and Setters (Omitted for brevity)
    }
}

Let’s understand a few details about the binding:

  • Type-safe binding (List and Map)Notice how the comma separated roles in the properties file are bound to a List of roles and the permissions are bound to a Map.
  • Duration SupportAlso, note how the duration properties are safely bound to the Duration types. This is so awesome. Spring Boot allows you to specify durations with the following units in the application.properties files –
    • ns for nanoseconds
    • us for microseconds
    • ms for milliseconds
    • s for seconds
    • m for minutes
    • h for hours
    • d for days  
    The default unit is milliseconds. So if you don’t specify any unit in the properties file, It will be considered as milliseconds.Note that, you can also override the unit using @DurationUnit annotation as we have done in the above POJO class.
  • Naming ConventionAll the kebab case property names (ex: upload-dir) are bound to the corresponding camel case fields (ex: uploadDir) in the POJO class.Note that the properties can also be specified in camel case. But using kebab case is recommended.

Enabling Configuration Properties

You need to explicitly register the properties classes using the @EnableConfigurationProperties annotation as shown in the following example.

package com.example.demo;

import com.example.demo.config.AppProperties;
import org.springframework.boot.SpringApplication;
import org.springframework.boot.autoconfigure.SpringBootApplication;
import org.springframework.boot.context.properties.EnableConfigurationProperties;

@SpringBootApplication
@EnableConfigurationProperties(AppProperties.class)
public class ConfigPropertiesDemoApplication {

	public static void main(String[] args) {
		SpringApplication.run(ConfigPropertiesDemoApplication.class, args);
	}
}

Alternatively, you could also add a @Component annotation to the AppProperties class and the binding would still work.

Injecting Configuration Properties in your Spring Beans

The @ConfigurationProperties classes are regular spring beans, and you can inject them in the same way as any other bean.

In the following example, I’ve written a sample API that retrieves app details from configuration properties and returns them to the client –

package com.example.demo.controller;

import com.example.demo.config.AppProperties;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.web.bind.annotation.GetMapping;
import org.springframework.web.bind.annotation.RestController;

import java.util.HashMap;
import java.util.Map;

@RestController
public class IndexController {

    // Injecting ConfigurationProperties in your Beans
    @Autowired
    private AppProperties appProperties;

    @GetMapping("/")
    public Map<String, String> getAppDetails() {
        Map<String, String> appDetails = new HashMap<>();
        appDetails.put("name", appProperties.getName());
        appDetails.put("description", appProperties.getDescription());

        return appDetails;
    }
}

@ConfigurationProperties Validation

You can validate configuration properties using javax.validation constraints like @NotNull@NotEmpty etc.

To enable validation, you just need to add Spring’s @Validated annotation to the @ConfigurationProperties class –

package com.example.demo.config;

import org.springframework.boot.context.properties.ConfigurationProperties;
import org.springframework.boot.convert.DurationUnit;
import org.springframework.validation.annotation.Validated;

import javax.validation.Valid;
import javax.validation.constraints.NotEmpty;
import javax.validation.constraints.NotNull;
import java.time.Duration;
import java.time.temporal.ChronoUnit;
import java.util.ArrayList;
import java.util.HashMap;
import java.util.List;
import java.util.Map;

@ConfigurationProperties(prefix = "app")
@Validated
public class AppProperties {
    @NotNull
    private String name;
    private String description;
    private String uploadDir;
    private Duration connectTimeout = Duration.ofMillis(1000);
    @DurationUnit(ChronoUnit.SECONDS)
    private Duration readTimeout = Duration.ofSeconds(30);
    @Valid
    private final Security security = new Security();

    // Getters and Setters (Omitted for brevity)

    public static class Security {
        private String username;
        private String password;
        @NotEmpty
        private List<String> roles = new ArrayList<>();
        private boolean enabled;
        private Map<String, String> permissions = new HashMap<>();

        // Getters and Setters (Omitted for brevity)
    }
}

Now, The Spring Boot application will throw a validation exception on startup, if the name property is null or the security.roles property is empty

Environment based (Profile specific) Configuration Properties

In addition to the standard application.properties file, Spring Boot also allows you to define profile-specific properties with the following naming convention –

application-{profile}.properties

The profile specific property files are loaded from the same location as the application.properties file, with profile-specific properties always overriding the default ones.

To illustrate this, Let’s define some profile-specific property files, and override some of the default properties –

  • application-dev.properties## Override properties for dev environment app.name=ConfigurationPropertiesDemoApp-DEVELOPMENT app.security.username=project-dev
  • application-staging.properties## Override properties for staging environment app.name=ConfigurationPropertiesDemoApp-STAGING app.security.username=project-staging app.security.password=C@ll1C0d3r
  • application-prod.properties## Override properties for prod environment app.name=ConfigurationPropertiesDemoApp-PRODUCTION app.security.username=project-prod app.security.password=C@ll1C0d3r

Now, we need to activate a spring profile so that the corresponding properties file is loaded by spring boot.

Activating a Spring Profile

You can set active profiles in many ways –

  1. Using application propertiesThe default application.properties file is always loaded by Spring Boot. You can set active profiles in the application.properties file by adding the following property –spring.profiles.active=staging After adding the above property, Spring Boot will load application-staging.properties file as well, and override properties with the new values specified there.
  2. Using command line argumentYou can set active profiles on startup by providing the spring.profiles.active command line argument like this –# Packaging the app mvn clean package -Dspring.profiles.active=staging # Running the packaged jar with `spring.profiles.active` argument java -jar -Dspring.profiles.active=staging target/config-properties-demo-0.0.1-SNAPSHOT.jar Moreover, Here is how you can set active profiles while running the application with spring-boot:run command –mvn spring-boot:run -Dspring.profiles.active=dev
  3. Using environment variableLastly, you can also set active profiles using the SPRING_PROFILES_ACTIVE environment variable-export SPRING_PROFILES_ACTIVE=prod

Running the application

Let’s run the application and access the sample Rest API to see configuration properties in action –

mvn spring-boot:run
$ curl http://localhost:8080
{"name":"ConfigurationPropertiesDemoApp","description":"ConfigurationPropertiesDemoApp is a spring boot app that demonstrates how to use external configuration properties"}

Running the application with active profiles set to prod

mvn spring-boot:run -Dspring.profiles.active=prod
$ curl http://localhost:8080
{"name":"ConfigurationPropertiesDemoApp-PRODUCTION","description":"ConfigurationPropertiesDemoApp-PRODUCTION is a spring boot app that demonstrates how to use external configuration properties"}

Conclusion

@ConfigurationProperties are a really nice way to bind external configurations in a type-safe manner. I use this feature in almost all my projects. Moreover, Spring Boot’s auto-configurations also rely on configuration properties.

Let me know what do you think about this feature in the comment section below.

Three Number Sum Solution
06
Feb
2021

Three Number Sum Solution

Three Number Sum Problem Statement

Given an array of integers, find all triplets in the array that sum up to a given target value.

In other words, given an array arr and a target value target, return all triplets a, b, c such that a + b + c = target.

Example:

Input array: [7, 12, 3, 1, 2, -6, 5, -8, 6]
Target sum: 0

Output: [[2, -8, 6], [3, 5, -8], [1, -6, 5]]

Three Number Sum Problem solution in Java

METHOD 1. Naive approach: Use three for loops

The naive approach is to just use three nested for loops and check if the sum of any three elements in the array is equal to the given target.

Time complexity: O(n^3)

import java.util.Scanner;
import java.util.List;
import java.util.ArrayList;
import java.util.Arrays;

class ThreeSum {

  // Time complexity: O(n^3)
  private static List<Integer[]> findThreeSum_BruteForce(int[] nums, int target) {
    List<Integer[]> result = new ArrayList<>();
    for (int i = 0; i < nums.length; i++) {
      for (int j = i + 1; j < nums.length; j++) {
        for (int k = j + 1; k < nums.length; k++) {
          if (nums[i] + nums[j] + nums[k] == target) {
            result.add(new Integer[] { nums[i], nums[j], nums[k] });
          }
        }
      }
    }
    return result;
  }

  public static void main(String[] args) {
    Scanner keyboard = new Scanner(System.in);

    int n = keyboard.nextInt();
    int[] nums = new int[n];

    for (int i = 0; i < n; i++) {
      nums[i] = keyboard.nextInt();
    }
    int target = keyboard.nextInt();

    keyboard.close();

    List<Integer[]> result = findThreeSum_Sorting(nums, target);

    for(Integer[] triplets: result) {
      for(int num: triplets) {
        System.out.print(num + " ");
      }
      System.out.println();
    }
  }
}

METHOD 2. Use Sorting along with the two-pointer sliding window approach

Another approach is to first sort the array, then –

  • Iterate through each element of the array and for every iteration,
    • Fix the first element (nums[i])
    • Try to find the other two elements whose sum along with nums[i] gives target. This boils down to the two sum problem.

Time complexity: O(n^2)

import java.util.Scanner;
import java.util.List;
import java.util.ArrayList;
import java.util.Arrays;

class ThreeSum {

  // Time complexity: O(n^2)
  private static List<Integer[]> findThreeSum_Sorting(int[] nums, int target) {
    List<Integer[]> result = new ArrayList<>();
    Arrays.sort(nums);
    for (int i = 0; i < nums.length; i++) {
      int left = i + 1;
      int right = nums.length - 1;
      while (left < right) {
        if (nums[i] + nums[left] + nums[right] == target) {
          result.add(new Integer[] { nums[i], nums[left], nums[right] });
          left++;
          right--;
        } else if (nums[i] + nums[left] + nums[right] < target) {
          left++;
        } else {
          right--;
        }
      }
    }
    return result;
  }
}

METHOD 3. Use a Map/Set

Finally, you can also solve the problem using a Map/Set. You just need to iterate through the array, fix the first element, and then try to find the other two elements using the approach similar to the two sum problem.

I’m using a Set in the following solution instead of a Map as used in the two-sum problem because in the two-sum problem, we had to keep track of the index of the elements as well. But In this problem, we just care about the element and not its index.

Time complexity: O(n^2)

import java.util.Set;
import java.util.Scanner;
import java.util.HashSet;

class ThreeSum {

  // Time complexity: O(n^2)
  private static List<Integer[]> findThreeSum(int[] nums, int target) {
    List<Integer[]> result = new ArrayList<>();
    for (int i = 0; i < nums.length; i++) {
      int currentTarget = target - nums[i];
      Set<Integer> existingNums = new HashSet<>();
      for (int j = i + 1; j < nums.length; j++) {
        if (existingNums.contains(currentTarget - nums[j])) {
          result.add(new Integer[] { nums[i], nums[j], currentTarget - nums[j] });
        } else {
          existingNums.add(nums[j]);
        }
      }
    }
    return result;
  }
}

Liked the Article? Share it on Social media!

Log4J Asynchronous Logging
03
Mar
2021

Log4J Asynchronous Logging

Asynchronous Logging

Log4j2 Supports Async loggers. These loggers provide a drastic improvement in performance compared to their synchronous counterparts.

Async Loggers internally use a library called Disruptor for asynchronous logging.

We need to include Disruptor dependency for using async loggers. Add the following to your pom.xml file –

<!-- Needed for Async Logging with Log4j 2 -->
<dependency>
    <groupId>com.lmax</groupId>
    <artifactId>disruptor</artifactId>
    <version>3.3.6</version>
</dependency>
How to get current Date and Time in Java
03
Mar
2021

How to get current Date and Time in Java

In this article, you’ll find several examples to get the current date, current time, current date & time, current date & time in a specific timezone in Java.

Get current Date in Java

import java.time.LocalDate;

public class CurrentDateTimeExample {
    public static void main(String[] args) {
        // Current Date
        LocalDate currentDate = LocalDate.now();
        System.out.println("Current Date: " + currentDate);
    }
}

Get current Time in Java

import java.time.LocalTime;

public class CurrentDateTimeExample {
    public static void main(String[] args) {
        // Current Time
        LocalTime currentTime = LocalTime.now();
        System.out.println("Current Time: " + currentTime);
    }
}

Get current Date and Time in Java

import java.time.LocalDateTime;

public class CurrentDateTimeExample {
    public static void main(String[] args) {
        // Current Date and Time
        LocalDateTime currentDateTime = LocalDateTime.now();
        System.out.println("Current Date & time: " + currentDateTime);
    }
}

Get current Date and Time in a specific Timezone in Java

import java.time.ZoneId;
import java.time.ZonedDateTime;

public class CurrentDateTimeExample {
    public static void main(String[] args) {
        // Current Date and Time in a given Timezone
        ZonedDateTime currentNewYorkDateTime = ZonedDateTime.now(ZoneId.of("America/New_York"));
        System.out.println(currentNewYorkDateTime);
    }
}
Convert a String to Date (LocalDate, LocalDateTime, ZonedDateTime, LocalTime) in Java
03
Mar
2021

Convert a String to Date (LocalDate, LocalDateTime, ZonedDateTime, LocalTime) in Java

In this article, you’ll find several examples demonstrating how to convert a date represented as a String to a Date, LocalDate, LocalDateTime, ZonedDateTime, or LocalTime instance in Java.

How to convert a String to Date in Java using SimpleDateFormat

Dates are represented as Strings by using some patterns and symbols. Java’s SimpleDateFormat class uses the following patterns and symbols for parsing a String to Date.

SimpleDateFormat Date and Time Patterns

Let’s see an example:

import java.text.ParseException;
import java.text.SimpleDateFormat;
import java.util.Date;

public class SimpleDateFormatExample {
    public static void main(String[] args) {
        SimpleDateFormat dateFormatter = new SimpleDateFormat("yyyy-MM-dd");
        String dateStr = "2020-01-31";

        try {
            // Parsing a String to Date
            Date date = dateFormatter.parse(dateStr);
            System.out.println(date);
        } catch (ParseException e) {
            e.printStackTrace();
        }
    }
}

Here is another example in which we parse a more complex date time representation:

import java.text.ParseException;
import java.text.SimpleDateFormat;
import java.util.Date;

public class SimpleDateFormatExample {
    public static void main(String[] args) {
        SimpleDateFormat dateTimeFormatter = new SimpleDateFormat("E, MMM dd yyyy, hh:mm:ss a");
        String dateTimeStr = "Fri, Jan 31 2020, 10:30:45 PM";

        try {
            // Parse the String representation of date and time to Date
            Date date = dateTimeFormatter.parse(dateTimeStr);
            System.out.println(date);
        } catch (ParseException e) {
            e.printStackTrace();
        }
    }
}

Convert/Parse a String to LocalDate

We can also use the DateTime API introduced in Java 8 to convert a String to an instance of various DateTime classes like LocalDate, LocalTime, LocalDateTime, ZonedDateTime etc.

The DateTime API has a DateTimeFormatter class that can be used to define the date time format and parse the String according to the specified date time format. The DateTimeFormatter class uses the following patterns and symbols:

DateTimeFormatter Patterns and Symbols

Let’s see an example of parsing a String to LocalDate:

import java.time.LocalDate;
import java.time.format.DateTimeFormatter;

public class LocalDateParseExample {
    public static void main(String[] args) {
        // Parse a String in ISO Date format (yyyy-MM-dd) to LocalDate
        LocalDate date1 = LocalDate.parse("2020-02-28");
        System.out.println(date1);

        // Parse a String in a custom date format to LocalDate using DateTimeFormatter
        DateTimeFormatter dateFormatter = DateTimeFormatter.ofPattern("dd/MM/yyyy");
        LocalDate date2 = LocalDate.parse("28/02/2020", dateFormatter);
        System.out.println(date2);

        // Parse a String in a custom date-time format to LocalDate using DateTimeFormatter
        DateTimeFormatter dateTimeFormatter = DateTimeFormatter.ofPattern("E, MMM dd yyyy, hh:mm:ss a");
        LocalDate date3 = LocalDate.parse("Fri, Feb 14 2020, 10:20:50 PM", dateTimeFormatter);
        System.out.println(date3);

    }
}

Convert/Parse a String to LocalDateTime

import java.time.LocalDateTime;
import java.time.format.DateTimeFormatter;

public class LocalDateTimeParseExample {
    public static void main(String[] args) {
        // Parse a String in ISO DateTime format (yyyy-MM-ddTHH:mm:ss) to LocalDateTime
        LocalDateTime dateTime1 = LocalDateTime.parse("2020-01-31T10:15:30");
        System.out.println(dateTime1);

        // Parse a String in a custom date format to LocalDate using DateTimeFormatter
        DateTimeFormatter dateTimeFormatter = DateTimeFormatter.ofPattern("MMM dd yyyy, HH:mm");
        LocalDateTime dateTime2 = LocalDateTime.parse("Jan 11 2020, 10:30", dateTimeFormatter);
        System.out.println(dateTime2);

        // Parse a String in a custom DateTime format to LocalDateTime using DateTimeFormatter
        DateTimeFormatter descriptiveDateTimeFormatter = DateTimeFormatter.ofPattern("E, MMM dd yyyy, hh:mm:ss a");
        LocalDateTime dateTime3 = LocalDateTime.parse("Fri, Feb 14 2020, 10:20:50 PM", descriptiveDateTimeFormatter);
        System.out.println(dateTime3);

    }
}

Convert/Parse a String to LocalTime

import java.time.LocalTime;
import java.time.format.DateTimeFormatter;

public class LocalTimeParseExample {
    public static void main(String[] args) {
        // Parse a String in ISO Time format (HH:mm:ss) to LocalDate
        LocalTime time1 = LocalTime.parse("12:30:50");
        System.out.println(time1);

        // Parse a String in a custom date format to LocalTime using DateTimeFormatter
        DateTimeFormatter dateTimeFormatter = DateTimeFormatter.ofPattern("dd/MM/yyyy HH:mm:ss");
        LocalTime time2 = LocalTime.parse("28/02/2020 16:45:30", dateTimeFormatter);
        System.out.println(time2);

    }
}

Convert/Parse a String to ZonedDateTime

import java.time.ZonedDateTime;
import java.time.format.DateTimeFormatter;

public class ZonedDateTimeParseExample {
    public static void main(String[] args) {
        // Parse a String in ISO DateTime format to ZonedDateTime
        ZonedDateTime dateTime1 = ZonedDateTime.parse("2020-01-31T10:15:30+01:00[Europe/Paris]");
        System.out.println(dateTime1);

        // Parse a String in a custom date time format to ZonedDateTime using DateTimeFormatter
        DateTimeFormatter dateTimeFormatter = DateTimeFormatter.ofPattern("MMM dd yyyy, HH:mm (VV)");
        ZonedDateTime dateTime2 = ZonedDateTime.parse("Jan 11 2020, 10:30 (America/Los_Angeles)", dateTimeFormatter);
        System.out.println(dateTime2);
    }
}
A Comparison Between Spring and Spring Boot
26
Mar
2021

A Comparison Between Spring and Spring Boot

What Is Spring?

Simply put, the Spring framework provides comprehensive infrastructure support for developing Java applications.

It’s packed with some nice features like Dependency Injection, and out of the box modules like:

  • Spring JDBC
  • Spring MVC
  • Spring Security
  • Spring AOP
  • Spring ORM
  • Spring Test

These modules can drastically reduce the development time of an application.

For example, in the early days of Java web development, we needed to write a lot of boilerplate code to insert a record into a data source. By using the JDBCTemplate of the Spring JDBC module, we can reduce it to a few lines of code with only a few configurations.

3. What Is Spring Boot?

Spring Boot is basically an extension of the Spring framework, which eliminates the boilerplate configurations required for setting up a Spring application.

It takes an opinionated view of the Spring platform, which paves the way for a faster and more efficient development ecosystem.

Here are just a few of the features in Spring Boot:

  • Opinionated ‘starter’ dependencies to simplify the build and application configuration
  • Embedded server to avoid complexity in application deployment
  • Metrics, Health check, and externalized configuration
  • Automatic config for Spring functionality – whenever possible

Let’s get familiar with both of these frameworks step by step.

4. Maven Dependencies

First of all, let’s look at the minimum dependencies required to create a web application using Spring:

<dependency>
    <groupId>org.springframework</groupId>
    <artifactId>spring-web</artifactId>
    <version>5.3.5</version>
</dependency>
<dependency>
    <groupId>org.springframework</groupId>
    <artifactId>spring-webmvc</artifactId>
    <version>5.3.5</version>
</dependency>

Unlike Spring, Spring Boot requires only one dependency to get a web application up and running:

<dependency>
    <groupId>org.springframework.boot</groupId>
    <artifactId>spring-boot-starter-web</artifactId>
    <version>2.4.4</version>
</dependency>

All other dependencies are added automatically to the final archive during build time.

Another good example is testing libraries. We usually use the set of Spring Test, JUnit, Hamcrest, and Mockito libraries. In a Spring project, we should add all of these libraries as dependencies.

Alternatively, in Spring Boot we only need the starter dependency for testing to automatically include these libraries.

Spring Boot provides a number of starter dependencies for different Spring modules. Some of the most commonly used ones are:

  • spring-boot-starter-data-jpa
  • spring-boot-starter-security
  • spring-boot-starter-test
  • spring-boot-starter-web
  • spring-boot-starter-thymeleaf

For the full list of starters, also check out the Spring documentation.

5. MVC Configuration

Let’s explore the configuration required to create a JSP web application using both Spring and Spring Boot.

Spring requires defining the dispatcher servlet, mappings, and other supporting configurations. We can do this using either the web.xml file or an Initializer class:

public class MyWebAppInitializer implements WebApplicationInitializer {
 
    @Override
    public void onStartup(ServletContext container) {
        AnnotationConfigWebApplicationContext context
          = new AnnotationConfigWebApplicationContext();
        context.setConfigLocation("com.baeldung");
 
        container.addListener(new ContextLoaderListener(context));
 
        ServletRegistration.Dynamic dispatcher = container
          .addServlet("dispatcher", new DispatcherServlet(context));
         
        dispatcher.setLoadOnStartup(1);
        dispatcher.addMapping("/");
    }
}

We also need to add the @EnableWebMvc annotation to a @Configuration class, and define a view-resolver to resolve the views returned from the controllers:

@EnableWebMvc
@Configuration
public class ClientWebConfig implements WebMvcConfigurer { 
   @Bean
   public ViewResolver viewResolver() {
      InternalResourceViewResolver bean
        = new InternalResourceViewResolver();
      bean.setViewClass(JstlView.class);
      bean.setPrefix("/WEB-INF/view/");
      bean.setSuffix(".jsp");
      return bean;
   }
}

By comparison, Spring Boot only needs a couple of properties to make things work once we add the web starter:

spring.mvc.view.prefix=/WEB-INF/jsp/
spring.mvc.view.suffix=.jsp

All of the Spring configuration above is automatically included by adding the Boot web starter through a process called auto-configuration.

What this means is that Spring Boot will look at the dependencies, properties, and beans that exist in the application and enable configuration based on these.

Of course, if we want to add our own custom configuration, then the Spring Boot auto-configuration will back away.

5.1. Configuring Template Engine

Now let’s learn how to configure a Thymeleaf template engine in both Spring and Spring Boot.

In Spring, we need to add the thymeleaf-spring5 dependency and some configurations for the view resolver:

@Configuration
@EnableWebMvc
public class MvcWebConfig implements WebMvcConfigurer {

    @Autowired
    private ApplicationContext applicationContext;

    @Bean
    public SpringResourceTemplateResolver templateResolver() {
        SpringResourceTemplateResolver templateResolver = 
          new SpringResourceTemplateResolver();
        templateResolver.setApplicationContext(applicationContext);
        templateResolver.setPrefix("/WEB-INF/views/");
        templateResolver.setSuffix(".html");
        return templateResolver;
    }

    @Bean
    public SpringTemplateEngine templateEngine() {
        SpringTemplateEngine templateEngine = new SpringTemplateEngine();
        templateEngine.setTemplateResolver(templateResolver());
        templateEngine.setEnableSpringELCompiler(true);
        return templateEngine;
    }

    @Override
    public void configureViewResolvers(ViewResolverRegistry registry) {
        ThymeleafViewResolver resolver = new ThymeleafViewResolver();
        resolver.setTemplateEngine(templateEngine());
        registry.viewResolver(resolver);
    }
}

Spring Boot 1 only requires the dependency of spring-boot-starter-thymeleaf to enable Thymeleaf support in a web application. Due to the new features in Thymeleaf3.0, we also have to add thymeleaf-layout-dialect as a dependency in a Spring Boot 2 web application. Alternatively, we can choose to add a spring-boot-starter-thymeleaf dependency that’ll take care of all of this for us.

Once the dependencies are in place, we can add the templates to the src/main/resources/templates folder and the Spring Boot will display them automatically.

6. Spring Security Configuration

For the sake of simplicity, we’ll see how the default HTTP Basic authentication is enabled using these frameworks.

Let’s start by looking at the dependencies and configuration we need to enable Security using Spring.

Spring requires both the standard spring-security-web and spring-security-config dependencies to set up Security in an application.

Next we need to add a class that extends the WebSecurityConfigurerAdapter and makes use of the @EnableWebSecurity annotation:

@Configuration
@EnableWebSecurity
public class CustomWebSecurityConfigurerAdapter extends WebSecurityConfigurerAdapter {
 
    @Autowired
    public void configureGlobal(AuthenticationManagerBuilder auth) throws Exception {
        auth.inMemoryAuthentication()
          .withUser("user1")
            .password(passwordEncoder()
            .encode("user1Pass"))
          .authorities("ROLE_USER");
    }
 
    @Override
    protected void configure(HttpSecurity http) throws Exception {
        http.authorizeRequests()
          .anyRequest().authenticated()
          .and()
          .httpBasic();
    }
    
    @Bean
    public PasswordEncoder passwordEncoder() {
        return new BCryptPasswordEncoder();
    }
}

Here we’re using inMemoryAuthentication to set up the authentication.

Spring Boot also requires these dependencies to make it work, but we only need to define the dependency of spring-boot-starter-security as this will automatically add all the relevant dependencies to the classpath.

The security configuration in Spring Boot is the same as the one above.

To see how the JPA configuration can be achieved in both Spring and Spring Boot, we can check out our article A Guide to JPA with Spring.

7. Application Bootstrap

The basic difference in bootstrapping an application in Spring and Spring Boot lies with the servlet. Spring uses either the web.xml or SpringServletContainerInitializer as its bootstrap entry point.

On the other hand, Spring Boot uses only Servlet 3 features to bootstrap an application. Let’s talk about this in detail.

7.1. How Spring Bootstraps?

Spring supports both the legacy web.xml way of bootstrapping as well as the latest Servlet 3+ method.

Let’s see the web.xml approach in steps:

  1. Servlet container (the server) reads web.xml.
  2. The DispatcherServlet defined in the web.xml is instantiated by the container.
  3. DispatcherServlet creates WebApplicationContext by reading WEB-INF/{servletName}-servlet.xml.
  4. Finally, the DispatcherServlet registers the beans defined in the application context.

Here’s how Spring bootstraps using the Servlet 3+ approach:

  1. The container searches for classes implementing ServletContainerInitializer and executes.
  2. The SpringServletContainerInitializer finds all classes implementing WebApplicationInitializer.
  3. The WebApplicationInitializer creates the context with XML or @Configuration classes.
  4. The WebApplicationInitializer creates the DispatcherServlet with the previously created context.

7.2. How Spring Boot Bootstraps?

The entry point of a Spring Boot application is the class which is annotated with @SpringBootApplication:

@SpringBootApplication
public class Application {
    public static void main(String[] args) {
        SpringApplication.run(Application.class, args);
    }
}

By default, Spring Boot uses an embedded container to run the application. In this case, Spring Boot uses the public static void main entry point to launch an embedded web server.

It also takes care of the binding of the Servlet, Filter, and ServletContextInitializer beans from the application context to the embedded servlet container.

Another feature of Spring Boot is that it automatically scans all the classes in the same package or sub packages of the Main-class for components.

Additionally, Spring Boot provides the option of deploying it as a web archive in an external container. In this case, we have to extend the SpringBootServletInitializer:

@SpringBootApplication
public class Application extends SpringBootServletInitializer {
    // ...
}

Here the external servlet container looks for the Main-class defined in the META-INF file of the web archive, and the SpringBootServletInitializer will take care of binding the Servlet, Filter, and ServletContextInitializer.

8. Packaging and Deployment

Finally, let’s see how an application can be packaged and deployed. Both of these frameworks support common package managing technologies like Maven and Gradle; however, when it comes to deployment, these frameworks differ a lot.

For instance, the Spring Boot Maven Plugin provides Spring Boot support in Maven. It also allows packaging executable jar or war archives and running an application “in-place.”

Some of the advantages of Spring Boot over Spring in the context of deployment include:

  • Provides embedded container support
  • Provision to run the jars independently using the command java -jar
  • Option to exclude dependencies to avoid potential jar conflicts when deploying in an external container
  • Option to specify active profiles when deploying
  • Random port generation for integration tests

9. Conclusion

In this article, we learned about the differences between Spring and Spring Boot.

In a few words, we can say that Spring Boot is simply an extension of Spring itself to make development, testing, and deployment more convenient.