Client Design

openGeminiJuly 1, 2024About 4 minAbout 1169 words

Background

Due to the InfluxDB 1.X client largely being in maintenance mode, and openGemini continuously evolving, the community has decided to develop its own client SDK for openGemini to better support it. This includes functionalities like supporting multiple server addresses and the Apache Arrow Flight protocol.

Client SDK Planned Features

  • Support for connecting to multiple server addresses
  • Support for the Apache Arrow Flight protocol
  • Capabilities for SQL queries, structured queries, writing, and batch writing are detailed in the UML diagram below
  • Default timeouts, with a connection timeout of 10 seconds and read/write timeout of 30 seconds

The methods described in this document assume that the programming language does not support overloading. If overloading is supported by the programming language, some optimizations and adjustments can be made to the method names.

Client constructor params design

contains
1
many
contains
contains
OpenGeminiClient
+ List<Address> addresses
+ AuthConfig authConfig // nullable, if null, means no auth
+ BatchConfig batchConfig // nullable, if null, means batch is disabled
+ timeout
+ connectTimeout
+ enum contentType // json, csv, msgpack
+ enum compressMethod // gzip, zstd, br
+ TlsConfig tlsConfig // nullable, language specific
+ GrpcConfig grpcConfig // if null, call WriteByGrpc will nothing to do, otherwise send write request by gRPC
+void close()
Address
+ String host
+ int Port // in rust, it is u16
AuthConfig
+ AuthType authType // enum Password, Token. The server currently does not support the Token type, and the SDK implementation for Token is incomplete.
+ String username
+ String password
+ String token
BatchConfig
+ Duration batchInterval // must be greater than 0
+ int batchSize // must be greater than 0, if set too large, may cause client overflow or server-side rejected the request.
GrpcConfig
+ List<Address> addresses
+ AuthConfig authConfig
+ BatchConfig batchConfig
+ enum compressMethod // gzip, zstd, br
+ TlsConfig tlsConfig
+ timeout

Database & RetentionPolicy management design

OpenGeminiClient
+void CreateDatabase(String database)
+void CreateDatabaseWithRp(String database, rpConfig RpConfig)
+String[] ShowDatabases()
+void DropDatabase(String database)
+void CreateRetentionPolicy(String database, RpConfig rpConfig, bool isDefault)
+void UpdateRetentionPolicy(String database, RpConfig rpConfig, bool isDefault)
+RetentionPolicy[] ShowRetentionPolicies(String database)
+void DropRetentionPolicy(String database, String retentionPolicy)
+void CreateMeasurement(CreateMeasurementBuilder builder)
+String[] ShowMeasurements(ShowMeasurementBuilder builder)
+void DropMeasurement(String database, String retentionPolicy, String measurement)
+Map[String]String[] ShowTagKeys(ShowTagKeysBuilder builder)
+String[] ShowTagValues(ShowTagValuesBuilder builder)
+Map[String]Map[String]String ShowFieldKeys(String database, Option measurement)
+String[] ShowSeries(ShowSeriesBuilder builder)
RpConfig
+ String Name // non-null
+ String Duration // non-null
+ String ShardGroupDuration // nullable
+ String IndexDuration // nullable
CreateMeasurementBuilder
+CreateMeasurementBuilder Tags(String[] tags)
+CreateMeasurementBuilder FieldMap(map[String]FieldType fields)
+CreateMeasurementBuilder ShardType(ShardType shardType)
+CreateMeasurementBuilder ShardKeys(String[] shardKeys)
+CreateMeasurementBuilder FullTextIndex()
+CreateMeasurementBuilder IndexList(String[] indexes)
+CreateMeasurementBuilder EngineType(EngineType engineType)
+CreateMeasurementBuilder PrimaryKey(String[] primaryKeys)
+CreateMeasurementBuilder SortKeys(String[] sortKeys)
+String build()
ShowMeasurementBuilder
+ShowMeasurementBuilder Filter(ComparisonOperator operator, String regex)
+String build()
«enum»
FieldType
Bool // BOOL
Int64 // INT64
Float64 // FLOAT64
String // STRING
«enum»
ShardType
Hash // HASH
Range // RANGE
«enum»
EngineType
ColumnStore // columnstore
ShowTagKeysBuilder
ShowTagKeysBuilder Database(String database)
ShowTagKeysBuilder Measurement(String measurement)
ShowTagKeysBuilder RetentionPolicy(String rp)
ShowTagKeysBuilder Limit(int limit)
ShowTagKeysBuilder Offset(int offset)
ShowTagValuesBuilder
ShowTagValuesBuilder Database(String database)
ShowTagValuesBuilder Measurement(String measurement)
ShowTagValuesBuilder RetentionPolicy(String rp)
ShowTagValuesBuilder Limit(int limit)
ShowTagValuesBuilder Offset(int offset)
ShowTagValuesBuilder With(String[] keys)
ShowTagValuesBuilder Where(String key, ComparisonOperator operator, String value)
ShowSeriesBuilder
ShowSeriesBuilder Database(String database)
ShowSeriesBuilder Measurement(String measurement)
ShowSeriesBuilder RetentionPolicy(String rp)
ShowSeriesBuilder Limit(int limit)
ShowSeriesBuilder Offset(int offset)
ShowTagValuesBuilder Where(String key, ComparisonOperator operator, String value)

Write point design

Query design

Query
+ String database
+ String retentionPolicy
+ String command
+ TimeFormat timeFormat
+ Precision precision
«enum»
Precision
PRECISIONMICROSECOND("u")
PRECISIONNANOSECOND("ns")
PRECISIONMILLISECOND("ms")
PRECISIONSECOND("s")
PRECISIONMINUTE("m")
PRECISIONHOUR("h")
«enum»
TimeFormat
RFC3339("rfc3339")
TIMESTAMP("timestamp")

QueryBuilder design

QueryBuilder
+static Create() : QueryBuilder
+Select(Expression[] selectExprs) : QueryBuilder
+From(String[] from) : QueryBuilder
+Where(Condition where) : QueryBuilder
+GroupBy(Expression[] groupByExpressions) : QueryBuilder
+OrderBy(order: SortOrder) : QueryBuilder
+Limit(limit: int64) : QueryBuilder
+Offset(offset: int64) : QueryBuilder
+Timezone(timezone: *time.Location) : QueryBuilder
+Build() : Query
«interface»
Expression
ConstantExpression
- Object value
StarExpression
FunctionExpression
- FunctionEnum function
- Expression[] arguments
AsExpression
- String alias
- Expression expression
ArithmeticExpression
- Expression Left
- Expression Right
- Operator ArithmeticOperator
«interface»
Condition
ComparisonCondition
- String column
- ComparisonOperator operator
- Object value
CompositeCondition
- LogicalOperator logicalOperator
- Condition[] conditions
«enum»
SortOrder
Asc
Desc
«enum»
ComparisonOperator
Equals
NotEquals
GreaterThan
LessThan
GreaterThanOrEquals
LessThanOrEquals
«enum»
LogicalOperator
And
Or
«enum»
FunctionEnum
Mean
Count
Sum
Min
Max
Time
FieldExpression
Query

Ping design

OpenGeminiClient
+Pong ping(int index) : // index selects one from multiple servers
Pong
+ String version

Inner Http client design

Using a design similar to InnerHttpClient, encapsulate authentication, load balancing, retry logic, and more within the internal structure, providing a simple interface to the client. This enhances modularity and code clarity.

InnerHttpClient
+void executeHttpGetByIdx(int idx, ...) : // specify server index
+void executeHttpRequestByIdx(int idx, String method, ...) : // specify server index
+void executeHttpGet(String method, ...) : // load balance
+void executeHttpRequest(String method, ...) : // load balance
-void executeHttpRequestInner(String url, String method, ...) : // inner method
executeHttpGetByIdx
executeHttpRequestByIdx
executeHttpRequestInner
executeHttpGet
executeHttpRequest

Error handling

Error message

Scene1 http request failed

$operation request failed, error: $error_details

Scene2 http response code is not 200~300

$operation error resp, code: $code, body: $body

Scene3 other error

$operation failed, error: $error_details
# example:
writePoint failed, unmarshall response body error: json: cannot unmarshal number ...
QueryResult
+ List<SeriesResult> results
+ String error
SeriesResult
+ List<Series> series // Series is an uncountable noun.
+ String error
Series
+ String name
Syntax error in graphmermaid version 10.0.2
OpenGeminiClient
+WritePoint(String database, Point point)
+WritePointWithRp(String database, String rp, Point point)
+WriteBatchPoints(String database, BatchPoints batchPoints)
+WriteBatchPointsWithRp(String database, String rp, BatchPoints batchPoints)
+WriteByGrpc(req WriteRequest) : // WriteRequest build from RecordBuilder
BatchPoints
+ List<Point> points
+AddPoint(Point)
Point
+ String measurement
+ Precision precision // enum, second, millisecond, microsecond, nanosecond, default is nanosecond
+ Time time // language specific
Syntax error in graphmermaid version 10.0.2