# Jodd HTTP

Developer-friendly HTTP library

**Jodd HTTP** is a tiny, raw HTTP client - and yet simple and convenient. It offers a developer-friendly way to send requests and read responses.

The goal is to provide a tiny layer on top of the existing Java sockets library.


# Installation

Tips on how to install Jodd HTTP library in your app

**Jodd HTTP** is released on Maven Central. You can use the following snippets to add it to your project:

{% tabs %}
{% tab title="Maven" %}

```markup
<dependency>
  <groupId>org.jodd</groupId>
  <artifactId>jodd-http</artifactId>
  <version>x.x.x</version>
</dependency>
```

{% endtab %}

{% tab title="Gradle" %}

```
implementation 'org.jodd:jodd-http:x.x.x'
```

{% endtab %}

{% tab title="Gradl.kt" %}

```kotlin
implementation("org.jodd:jodd-http:x.x.x")
```

{% endtab %}

{% tab title="Scala SBT" %}

```scala
libraryDependencies += "org.jodd" % "jodd-http" % "x.x.x"
```

{% endtab %}

{% tab title="Ivy" %}

```markup
<dependency org="org.jodd" name="jodd-http" rev="x.x.x" />
```

{% endtab %}

{% tab title="Leiningen" %}

```
[org.jodd/jodd-http "x.x.x"]
```

{% endtab %}

{% tab title="Buildr" %}

```
'org.jodd:jodd-http:jar:x.x.x'
```

{% endtab %}
{% endtabs %}

That is all!

### Snapshots

**Jodd HTTP** snapshots are published on [Maven Central Snapshot repo](https://oss.sonatype.org/content/repositories/snapshots/org/jodd/jodd-lagarto/).

{% hint style="warning" %}
Snapshots are released manually. Feel free to contact me if you need a new SNAPSHOT release sooner.
{% endhint %}


# Using the Client

### Simple GET request

```java
HttpRequest httpRequest = HttpRequest.get("http://jodd.org");
HttpResponse response = httpRequest.send();

System.out.println(response);
```

All **HTTP** classes offer a *fluent interface*, too; so you can write:

```java
HttpResponse response = HttpRequest.get("http://jodd.org").send();

System.out.println(response);
```

You can build the request step by step:

```java
HttpRequest request = new HttpRequest();
request
    .method("GET")
    .protocol("http")
    .host("srv")
    .port(8080)
    .path("/api/jsonws/user/get-user-by-id");
```

### Reading Response

When an HTTP request is sent, the whole response is stored in the `HttpResponse` instance. You can use the response for various stuff: read the `statusCode()` or `statusPhrase()`; or any header attribute.

A common thing is how to read the received response body. You may use one of the following methods:

* `bodyRaw()` - raw body content, always in `ISO-8859-1` encoding.
* `bodyText()` - body text, i.e. string encoded as specified by `Content-Type` header.
* `bodyBytes()` - returns the raw body as a byte array, so e.g. downloaded file

  can be saved.

The character encoding used in `bodyText()` is one set in the response headers. If the response does not specify the encoding in its headers (but e.g. only on the HTML page), you *must* specify the encoding with `charset()` the method before calling `bodyText()`. {: .attn}

### Query parameters

Query parameters may be specified in the URL line (but then they have to be encoded correctly):

```java
HttpResponse response = HttpRequest
    .get("http://srv:8080/api/user/get-user-by-id?userId=10194")
    .send();
```

A better and recommended way is with the `query()` method:

```java
HttpResponse response = HttpRequest
    .get("http://srv:8080/api/user/get-user-by-id")
    .query("userId", "10194")
    .send();
```

You can use `query()` for each parameter, or pass many arguments in one call (varargs). You can also provide `Map<String, String>` as a parameter too.

{% hint style="info" %}
Query parameters (as well as headers and form parameters) can be duplicated. Therefore, they are stored in an array internally. Use method `removeQuery` to remove some parameters or overloaded methods to replace a parameter.
{% endhint %}

Finally, you can reach the internal query map, that actually holds all parameters:

```java
Map<String, Object[]> httpParams = request.query();
httpParams.put("userId", new String[] {"10194"});
```

### Authentication

Basic authentication is made easy:

```java
request.basicAuthentication("user", "password");
```

Token-based authentication:

```java
request.tokenAuthentication("M4ORM....");
```

### POST and form parameters

```java
HttpResponse response = HttpRequest
    .post("http://srv:8080/api/jsonws/user/get-user-by-id")
    .form("userId", "10194")
    .send();
```

Use `form()` in the same way as `query()` to specify form parameters. Everything that is said for `query()` applies to the `form()`.

### Upload files

Again, it's easy: just add the file form parameter. Here is one real-world example:

```java
HttpRequest httpRequest = HttpRequest
    .post("http://srv:8080/api/dlapp/add-file-entry")
    .form(
        "repositoryId", "10178",
        "folderId", "11219",
        "sourceFileName", "a.zip",
        "mimeType", "application/zip",
        "title", "test",
        "description", "Upload test",
        "changeLog", "testing...",
        "file", new File("d:\\a.jpg.zip")
    );

HttpResponse httpResponse = httpRequest.send();
```

#### Monitor upload progress

When uploading a large file, it is helpful to monitor the progress. For that purpose, you can use `HttpProgressListener` like this:

```java
HttpResponse response = HttpRequest
    .post("http://localhost:8081/hello")
    .form("file", file)
    .monitor(new HttpProgressListener() {
        @Override
        public void transferred(long len) {
            System.out.println(len/size);
        }
    })
    .send();
```

Before the upload starts, `HttpProgressListener` calculates the `callbackSize` - the size of the chunk in bytes that will be transferred. By default, this size equals `1%` of the total size. Moreover, it is never less than `512` bytes.

`HttpProgressListener` contains the inner field `size` with the total size of the request. Note that this is the size of the whole request, not only the files! This is the actual number of bytes that are going to be sent, and it is always a bit larger than file size (due to protocol overhead).

### Headers

Add or reach header parameters with the method `header()`. Some common header parameters are already defined as methods, so you will find `contentType()` etc.

There are some shortcut methods that are commonly used:

* `contentTypeJson()` - specifies JSON content type
* `acceptJson()` - accepts JSON content.

### GZipped content

Just `unzip()` the response.

```java
HttpResponse response = HttpRequest
    .get("http://jodd.org")
    .acceptEncoding("gzip")
    .send();

System.out.println(response.unzip());
```

The `unzip()` method is safe; it will not fail if the response is not zipped.

### Set the body

You can set the request body manually:

```java
HttpResponse response = HttpRequest
    .get("http://srv:8080/api/jsonws/invoke")
    .body("{'a':1 23, 'b': 'hi'}")
    .basicAuthentication("test", "test")
    .send();
```

{% hint style="warning" %}
Setting the body discards all previously set `form()` parameters.&#x20;
{% endhint %}

### Charsets and Encodings

By default, query and form parameters are encoded in UTF-8.

```java
    HttpResponse response = HttpRequest
        .get("http://server/index.html")
        .queryEncoding("CP1251")
        .query("param", "value")
        .send();
```

You can set form encoding similarly. Moreover, form posting detects the value of **charset** in the "Content-Type" header, and if present, it will be used.

With received content, `body()` method always returns the **raw** string (encoded as ISO-8859-1). To get the string in usable form, use the method `bodyText()`. This method uses a provided **charset** from the "Content-Type" header and encodes the body string.

### Following redirection

By default `HttpRequest` does not follow redirection response. This can be changed by setting the `followRedirects(true)`. Now redirect responses are followed. When redirection is enabled, the original URL will NOT be preserved in the request object!

### Asynchronous sending

When `send()` is called, the program blocks until the response is received. By using `sendAsync()` the execution of the sending is passed to Javas fork-join pool, and will be executed asynchronously. Method returns `CompletableFuture<Response>`.


# Connection

### HttpConnection

Socket HTTP communication is encapsulated by `HttpConnection` interface. On `send()`, **Jodd HTTP** will `open()` connection if not already opened. HTTP connections are created by the connection provider instance: `HttpConnectionProvider`. The default connection provider is socket-based and it always returns a new `SocketHttpConnection` instance - that simply wraps a `Socket` and opens it.

It is common to use custom `HttpConnectionProvider`, based on default implementation. For example, you may extend the `SocketHttpConnectionProvider` and override `createSocket()` method to return sockets from some pool, or sockets with a different timeout.

Alternatively, you may even provide an instance of `HttpConnection` directly, without any provider.

### Working with Sockets

As said, the default communication goes through the plain `Socket`. Since it is a common need to tweak socket behavior before sending data, here are two ways of how you can do it with **Jodd HTTP**.

#### SocketHttpConnection

Since we know the default type of `HttpConnection`, we can simply get the instance after explicitly calling the `open()` and cast it:

```java
HttpRequest request = HttpRequest.get()...;
request.open();

SocketHttpConnection httpConnection =
    (SocketHttpConnection) request.httpConnection();
Socket socket = httpConnection.getSocket();
socket.setSoTimeout(1000);

...

HttpResponse response = request.send();
```

#### SocketHttpConnectionProvider

The other way is to use custom `HttpConnectionProvider` based on `SocketHttpConnectionProvider`. So you may create your own provider like this:

```java
public class MyConnectionProvider extends SocketHttpConnectionProvider {
    protected Socket createSocket(
            SocketFactory socketFactory, String host, int port)
            throws IOException {
        Socket socket = super.createSocket(socketFactory, host, port);
        socket.setSoTimeout(1000);
        return socket;
    }
}
```

The custom provider is set by `withConnectionProvider()`:

```java
HttpResponse response = HttpRequest
    .get()
    .withConnectionProvider(new MyConnectionProvider())
    ...
    send();
```

Alternatively, you can explicitly open a connection with the `open()` method:

```java
HttpConnectionProvider connectionProvider = new MyConnectionProvider();
...
HttpRequest request = HttpRequest.get()...;
HttpResponse response = request.open(connectionProvider).send();
```

{% hint style="danger" %}
Once when a connection is open by `open()` method, you can not alter it via the **Jodd HTTP** interface. For example, setting the timeouts *after* the open will have no effect.
{% endhint %}

### Keep-Alive

By default, all connections are marked as *closed*, to keep servers happy. **Jodd HTTP** allows usage of permanent connections through the keep-alive mode. The `HttpConnection` is opened on the first request and then re-used in communication session; the socked is not opened again if not needed and therefore it is reused for several requests.

There are several ways how to do this. The easiest way is the following:

```java
HttpRequest request = HttpRequest.get("http://jodd.org");
HttpResponse response = request.connectionKeepAlive(true).send();

// next request
request = HttpRequest.get("http://jodd.org/jodd.css");
response = request.keepAlive(response, true).send();

...

// last request
request = HttpRequest.get("http://jodd.org/jodd.png");
response = request.keepAlive(response, false).send();

// optionally
//response.close();
```

This example fires several requests over the same `HttpConnection` (i.e. the same socket). When in 'keep-alive' mode, *HTTP* continues using the existing connection, while paying attention to server responses. If the server explicitly requires a connection to be closed, *HTTP* will close it and then it will open a new connection to continue your session. You don't have to worry about this, just keep calling `keepAlive()` and it will magically do everything for you in the background. Just don't forget to pass `false` argument to the last call to indicate the server that is the last connection and that we want to close after receiving the last response. (if for some reasons the server does not respond correctly, you may close communication on the client-side with an explicit call to `response.close()`). One more thing - if a new connection has to be opened during this persistent session (when e.g. keep-alive max counter is finished or timeout expired) the same connection provider will be used as for the initial, first connection.

### Proxy

`HttpConnectionProvider` also allows you to specify the proxy. Just provide the `ProxyInfo` instance with the information about the used proxy (type, address, port, username, password):

```java
    SocketHttpConnectionProvider s = new SocketHttpConnectionProvider();
    s.useProxy(ProxyInfo.httpProxy("proxy_url", 1090, null, null));

    HttpResponse response = HttpRequest
        .get("http://jodd.org/")
        .withConnectionProvider(s)
        .send();
```

**Jodd HTTP** supports HTTP, SOCKS4, and SOCKE5 proxy types.

### Parse from InputStreams

Both `HttpRequest` and `HttpResponse` have a method `readFrom(InputStream)`. Basically, you can parse the input stream with these methods. This is, for example, how you can read the request on server-side.


# HttpSession

Sending requests and receiving responses is not enough when we have to emulate browsing through a website. For example, we might need to login, capture and carry-on the cookies to preserve the session, follow the redirects, make requests to get dynamic content... in the same way as a real web browser.

`HttpSession` is a tool just for that. It sends requests for you, handles 301 and 302 redirections automatically, reads and preserves cookies across the requests, and so on.

Usage is simple:

```java
HttpSession session = new HttpSession();

HttpRequest request = HttpRequest.get("www.facebook.com");
session.sendRequest(request);

// request is sent, and response is received

// process the HTML page
String page = session.getPage();

// create a new request
HttpRequest newRequest = HttpRequest.post(formAction);

session.sendRequest(newRequest);
```

`HttpSession` instance handles all the cookies, allowing the session to be tracked while browsing using HTTP and supports keep-alive persistent connections.


# HttpTunnel

**Jodd HTTP** is so flexible that you can easily build a HTTP tunnel with it - a small proxy between you and destination. We even give you a base class: `HttpTunnel` class, that provides easy HTTP tunneling. It opens the server socket on one port and tunnels the whole HTTP traffic to some target address.

[TinyTunnel](https://github.com/igr/tiny-tunnel) is one implementation that simply prints out the whole communication to the console.


# FAQ

Everything you wanted to know.

### How to save a binary file?

```java
final String link =
    "https://repo1.maven.org/maven2" +
    "/org/jodd/jodd-http/3.9.1/jodd-http-3.9.1.jar";

HttpResponse response = HttpRequest
    .get(link)
    .send();

byte[] bytes = response.bodyBytes();

FileUtil.writeBytes(
    new File(SystemUtil.userHome(), "jodd-http.jar"), bytes);
```

### How to follow multi redirects?

Either use `HttpBrowser`:

```java
HttpBrowser browser = new HttpBrowser();

browser.sendRequest(HttpRequest.get("google.com"));

// read response
Response response = browser.getResponse();
String page = browser.getPage();
```

or the flag `followRedirects()` to enable the following redirects:

```java
HttpResponse response =
    HttpRequest
        .get("google.com")
        .followRedirects(true)
        .send();
```

### What is the difference between HttpRequest and HttpBrowser?

`HttpRequest` represents just a single request; clean and simple.

`HttpBrowser` emulates browsing of a website (i.e. set of URLs) like a browser. Besides sending requests, it also stores and resends cookies, maintaining the current user session. Moreover, the `HttpBrowser` uses new request on redirection following, allows common request headers for all the requests etc.

### Server choses TLSv1.2, but that protocol version is not enabled?

Just add the following property `-Dhttps.protocols=TLSv1.1,TLSv1.2` which configures the JVM to specify which TLS protocol version should be used during HTTPS connections.

Since 8u292, JDK 11.0.11, JDK16+ Java requires use of TLS v1.2 or v1.3.

### SOCKS5: proxy returned 1

This is a common error when using Socks5 behind some VPNs.


# Contact

Let's keep in touch!

{% hint style="success" %}

## <info@jodd.org>

{% endhint %}