Java Client (V2)
Java client library to communicate with a DB server through its protocols. The current implementation only supports the HTTP interface. The library provides its own API to send requests to a server. The library also provides tools to work with different binary data formats (RowBinary & Native).
Setup
- Maven Central (project web page): https://mvnrepository.com/artifact/com.clickhouse/client-v2
- Nightly builds (repository link): https://s01.oss.sonatype.org/content/repositories/snapshots/com/clickhouse/
- Maven
- Gradle (Kotlin)
- Gradle
<dependency>
<groupId>com.clickhouse</groupId>
<artifactId>client-v2</artifactId>
<version>0.6.5</version>
</dependency>
// https://mvnrepository.com/artifact/com.clickhouse/client-v2
implementation("com.clickhouse:client-v2:0.6.5")
// https://mvnrepository.com/artifact/com.clickhouse/client-v2
implementation 'com.clickhouse:client-v2:0.6.5'
Initialization
The Client object is initialized by com.clickhouse.client.api.Client.Builder#build(). Each client has its own context and no objects are shared between them.
The Builder has configuration methods for convenient setup.
Example:
Client client = new Client.Builder()
.addEndpoint("https://clickhouse-cloud-instance:8443/")
.setUsername(user)
.setPassword(password)
.build();
Client is AutoCloseable and should be closed when not needed anymore.
Configuration
All settings are defined by instance methods (a.k.a configuration methods) that make the scope and context of each value clear. Major configuration parameters are defined in one scope (client or operation) and do not override each other.
Configuration is defined during client creation. See com.clickhouse.client.api.Client.Builder.
Common Definitions
ClickHouseFormat
Enum of supported formats. It includes all formats that ClickHouse supports.
raw- user should transcode raw datafull- the client can transcode data by itself and accepts a raw data stream-- operation not supported by ClickHouse for this format
This client version supports:
Insert API
insert(String tableName, InputStream data, ClickHouseFormat format)
Accepts data as an InputStream of bytes in the specified format. It is expected that data is encoded in the format.
Signatures
CompletableFuture<InsertResponse> insert(String tableName, InputStream data, ClickHouseFormat format, InsertSettings settings)
CompletableFuture<InsertResponse> insert(String tableName, InputStream data, ClickHouseFormat format)
Parameters
tableName - a target table name.
data - an input stream of an encoded data.
format - a format in which the data is encoded.
settings - request settings.
Return value
Future of InsertResponse type - result of the operation and additional information like server side metrics.
Examples
try (InputStream dataStream = getDataStream()) {
try (InsertResponse response = client.insert(TABLE_NAME, dataStream, ClickHouseFormat.JSONEachRow,
insertSettings).get(3, TimeUnit.SECONDS)) {
log.info("Insert finished: {} rows written", response.getMetrics().getMetric(ServerMetrics.NUM_ROWS_WRITTEN).getLong());
} catch (Exception e) {
log.error("Failed to write JSONEachRow data", e);
throw new RuntimeException(e);
}
}
insert(String tableName, List<?> data, InsertSettings settings)
Sends a write request to database. The list of objects is converted into an efficient format and then is sent to a server. The class of the list items should be registed up-front using register(Class, TableSchema) method.
Signatures
client.insert(String tableName, List<?> data, InsertSettings settings)
client.insert(String tableName, List<?> data)
Parameters
tableName - name of the target table.
data - collection DTO (Data Transfer Object) objects.
settings - request settings.
Return value
Future of InsertResponse type - the result of the operation and additional information like server side metrics.
Examples
// Important step (done once) - register class to pre-compile object serializer according to the table schema.
client.register(ArticleViewEvent.class, client.getTableSchema(TABLE_NAME));
List<ArticleViewEvent> events = loadBatch();
try (InsertResponse response = client.insert(TABLE_NAME, events).get()) {
// handle response, then it will be closed and connection that served request will be released.
}
InsertSettings
Configuration options for insert operations.
Configuration methods
- setQueryId(String queryId)
- Sets query ID that will be assigned to the operation
- setDeduplicationToken(String token)
- Sets the deduplication token. This token will be sent to the server and can be used to identify the query.
- waitEndOfQuery(Boolean waitEndOfQuery)
- Requests the server to wait for the and of the query before sending response.
- setInputStreamCopyBufferSize(int size)
- Copy buffer size. The buffer is used during write operations to copy data from user provided input stream to an output stream.
InsertResponse
Response object that holds result of insert operation. It is only available if the client got response from a server.
This object should be closed as soon as possible to release a connection because the connection cannot be re-used until all data of previous response is fully read.
- OperationMetrics getMetrics()
- Returns object with operation metrics
- String getQueryId()
- Returns query ID assigned for the operation by application (thru operation settings or by server).
Query API
query(String sqlQuery)
Sends sqlQuery as is. Response format is set by query settings. QueryResponse will hold a reference to the response stream that should be consumed by a reader for the supportig format.
Signatures
CompletableFuture<QueryResponse> query(String sqlQuery, QuerySettings settings)
CompletableFuture<QueryResponse> query(String sqlQuery)
Parameters
sqlQuery - a single SQL statement. The Query is sent as is to a server.
settings - request settings.
Return value
Future of QueryResponse type - a result dataset and additional information like server side metrics. The Response object should be closed after consuming the dataset.
Examples
final String sql = "select * from " + TABLE_NAME + " where title <> '' limit 10";
// Default format is RowBinaryWithNamesAndTypesFormatReader so reader have all information about columns
try (QueryResponse response = client.query(sql).get(3, TimeUnit.SECONDS);) {
// Create a reader to access the data in a convenient way
ClickHouseBinaryFormatReader reader = client.newBinaryFormatReader(response);
while (reader.hasNext()) {
reader.next(); // Read the next record from stream and parse it
// get values
double id = reader.getDouble("id");
String title = reader.getString("title");
String url = reader.getString("url");
// collecting data
}
} catch (Exception e) {
log.error("Failed to read data", e);
}
// put business logic outside of the reading block to release http connection asap.
query(String sqlQuery, Map<String, Object> queryParams, QuerySettings settings)
Sends sqlQuery as is. Additionally will send query parameters so the server can compile the SQL expression.
Signatures
CompletableFuture<QueryResponse> query(String sqlQuery, Map<String, Object> queryParams, QuerySettings settings)
Parameters
sqlQuery - sql expression with placeholders {}.
queryParams - map of variables to complete the sql expression on server.
settings - request settings.
Return value
Future of QueryResponse type - a result dataset and additional information like server side metrics. The Response object should be closed after consuming the dataset.
Examples
// define parameters. They will be sent to the server along with the request.
Map<String, Object> queryParams = new HashMap<>();
queryParams.put("param1", 2);
try (QueryResponse queryResponse =
client.query("SELECT * FROM " + table + " WHERE col1 >= {param1:UInt32}", queryParams, new QuerySettings()).get()) {
// Create a reader to access the data in a convenient way
ClickHouseBinaryFormatReader reader = client.newBinaryFormatReader(response);
while (reader.hasNext()) {
reader.next(); // Read the next record from stream and parse it
// reading data
}
} catch (Exception e) {
log.error("Failed to read data", e);
}
queryAll(String sqlQuery)
Queries a data in RowBinaryWithNamesAndTypes format. Returns the result as a collection. Read performance is the same as with the reader but more memory is required to hold the whole dataset.
Signatures
List<GenericRecord> queryAll(String sqlQuery)
Parameters
sqlQuery - sql expression to query data from a server.
Return value
Complete dataset represented by a list of GenericRecord objects that provide access in row style for the result data.
Examples
try {
log.info("Reading whole table and process record by record");
final String sql = "select * from " + TABLE_NAME + " where title <> ''";
// Read whole result set and process it record by record
client.queryAll(sql).forEach(row -> {
double id = row.getDouble("id");
String title = row.getString("title");
String url = row.getString("url");
log.info("id: {}, title: {}, url: {}", id, title, url);
});
} catch (Exception e) {
log.error("Failed to read data", e);
}
QuerySettings
Configuration options for query operations.
Configuration methods
- setQueryId(String queryId)
- Sets query ID that will be assigned to the operation
- setFormat(ClickHouseFormat format)
- Sets response format. See `RowBinaryWithNamesAndTypes` for the full list.
- setMaxExecutionTime(Integer maxExecutionTime)
- Sets operation execution time on server. Will not affect read timeout.
- waitEndOfQuery(Boolean waitEndOfQuery)
- Requests the server to wait for the and of the query before sending response.
- setUseServerTimeZone(Boolean useServerTimeZone)
- Server timezone (see client config) will be used to parse date/time types in the result of an operation. Default `false`
- setUseTimeZone(String timeZone)
- Requests server to use `timeZone` for time conversion. See session_timezone.
QueryResponse
Response object that holds result of query execution. It is only available if the client got a response from a server.
This object should be closed as soon as possible to release a connection because the connection cannot be re-used until all data of previous response is fully read.
- ClickHouseFormat getFormat()
- Returns a format in which data in the response is encoded.
- InputStream getInputStream()
- Returns uncompressed byte stream of data in the specified format.
- OperationMetrics getMetrics()
- Returns object with operation metrics
- String getQueryId()
- Returns query ID assigned for the operation by application (thru operation settings or by server).
- TimeZone getTimeZone()
- Returns timezone that should be used for handling Date/DateTime types in the response.
Examples
- Example code is available in repo
- Reference Spring Service implementation
Common API
getTableSchema(String table)
Fetches table schema for the table.
Signatures
TableSchema getTableSchema(String table)
TableSchema getTableSchema(String table, String database)
Parameters
table - table name for which schema data should be fetched.
database - database where the target table is defined.
Return value
Returns a TableSchema object with list of table columns.
getTableSchemaFromQuery(String sql)
Fetches schema from a SQL statement.
Signatures
TableSchema getTableSchemaFromQuery(String sql)
Parameters
sql - "SELECT" SQL statement which schema should be returned.
Return value
Returns a TableSchema object with columns matching the sql expression.
TableSchema
register(Class<?> clazz, TableSchema schema)
Compiles SerDe layer for the Java Class to use for writing/reading data with schema. The method will create a serializer and deserializer for the pair getter/setter and corresponding column.
Column match is found by extracting its name from a method name. For example, getFirstName will be for the column first_name or firstname.
Signatures
void register(Class<?> clazz, TableSchema schema)
Parameters
clazz - Class representing the POJO used to read/write data.
schema - Data schema to use for matching with POJO properties.
Examples
client.register(ArticleViewEvent.class, client.getTableSchema(TABLE_NAME));
Usage Examples
Complete examples code is stored in the repo in a 'example` folder:
- client-v2 - main set of examples.
- demo-service - example of how to use the client in a Spring Boot application.
- demo-kotlin-service - example of how to use the client in Ktor (Kotlin) application.