Get a DBaaS Kafka service

GET /dbaas-kafka/{name}

Path parameters

  • name string Required

    Service name

    Minimum length is 0, maximum length is 63.

Responses

  • 200 application/json

    200

    Hide response attributes Show response attributes object
    • updated-at string(date-time)

      Service last update timestamp (ISO 8601)

    • Kafka authentication methods

      Hide authentication-methods attributes Show authentication-methods attributes
      • Whether certificate/SSL authentication is enabled

      • sasl boolean

        Whether SASL authentication is enabled

    • node-count integer(int64)

      Number of service nodes in the active plan

      Minimum value is 0.

    • Kafka connection information properties

      Hide connection-info attributes Show connection-info attributes
    • node-cpu-count integer(int64)

      Number of CPUs for each node

      Minimum value is 0.

    • Whether Kafka REST is enabled

    • integrations array[object]

      Service integrations

      Hide integrations attributes Show integrations attributes object
    • zone string

      The zone where the service is running

    • node-states array[object]

      Automatic maintenance settings

      Hide node-states attributes Show node-states attributes object

      Automatic maintenance settings

      • name string Required

        Name of the service node

      • progress-updates array[object]

        Extra information regarding the progress for current state

        Hide progress-updates attributes Show progress-updates attributes object

        Extra information regarding the progress for current state

        • completed boolean Required

          Indicates whether this phase has been completed or not

        • current integer(int64)

          Current progress for this phase. May be missing or null.

          Minimum value is 0.

        • max integer(int64)

          Maximum progress value for this phase. May be missing or null. May change.

          Minimum value is 0.

        • min integer(int64)

          Minimum progress value for this phase. May be missing or null.

          Minimum value is 0.

        • phase string Required

          Key identifying this phase

          Values are stream, basebackup, prepare, or finalize.

        • unit string

          Unit for current/min/max values. New units may be added. If null should be treated as generic unit

      • role string

        Role of this node. Only returned for a subset of service types

        Values are standby, master, or read-replica.

      • state string Required

        Current state of the service node

        Values are leaving, running, syncing_data, setting_up_vm, or unknown.

    • name string Required

      Service name

      Minimum length is 0, maximum length is 63.

    • Whether Kafka Connect is enabled

    • type string Required

      DbaaS service name

      Minimum length is 0, maximum length is 64.

    • state string

      State of the service

      Values are running, rebuilding, rebalancing, or poweroff.

    • ip-filter array[string]

      Allow incoming connections from CIDR address block, e.g. '10.20.0.0/16'

    • Schema Registry configuration

      Hide schema-registry-settings attributes Show schema-registry-settings attributes
      • If true, Karapace / Schema Registry on the service nodes can participate in leader election. It might be needed to disable this when the schemas topic is replicated to a secondary cluster and Karapace / Schema Registry there must not participate in leader election. Defaults to true.

      • The durable single partition topic that acts as the durable log for the data. This topic must be compacted to avoid losing data due to retention policy. Please note that changing this configuration in an existing Schema Registry / Karapace setup leads to previous schemas being inaccessible, data encoded with them potentially unreadable and schema ID sequence put out of order. It's only possible to do the switch while Schema Registry / Karapace is disabled. Defaults to _schemas.

        Minimum length is 1, maximum length is 249. Format should match the following pattern: ^(?!\.$|\.\.$)[-_.A-Za-z0-9]+$.

    • backups array[object]

      List of backups for the service

      Hide backups attributes Show backups attributes object

      List of backups for the service

      • backup-name string Required

        Internal name of this backup

      • backup-time string(date-time) Required

        Backup timestamp (ISO 8601)

      • data-size integer(int64) Required

        Backup's original size before compression

        Minimum value is 0.

    • Kafka REST configuration

      Hide kafka-rest-settings attributes Show kafka-rest-settings attributes
      • Specify the default compression type for producers. This configuration accepts the standard compression codecs ('gzip', 'snappy', 'lz4', 'zstd'). It additionally accepts 'none' which is the default and equivalent to no compression.

        Values are gzip, snappy, lz4, zstd, or none.

      • If true, validate that given schema is registered under expected subject name by the used name strategy when producing messages.

        Default value is true.

      • Name strategy to use when selecting subject for storing schemas

        Values are topic_name, record_name, or topic_record_name. Default value is topic_name.

      • If true the consumer's offset will be periodically committed to Kafka in the background

        Default value is true.

      • The number of acknowledgments the producer requires the leader to have received before considering a request complete. If set to 'all' or '-1', the leader will wait for the full set of in-sync replicas to acknowledge the record.

        Values are all, -1, 0, or 1. Default value is 1.

      • Maximum number of bytes in unencoded message keys and values by a single request

        Minimum value is 0, maximum value is 671088640. Default value is 67108864.

      • The maximum size of a request in bytes. Note that Kafka broker can also cap the record batch size.

        Minimum value is 0, maximum value is 2147483647. Default value is 1048576.

      • Maximum number of SimpleConsumers that can be instantiated per broker

        Minimum value is 10, maximum value is 250. Default value is 25.

      • Wait for up to the given delay to allow batching records together

        Minimum value is 0, maximum value is 5000. Default value is 0.

      • The maximum total time to wait for messages for a request if the maximum number of messages has not yet been reached

        Minimum value is 1000, maximum value is 30000. Values are 1000, 15000, or 30000. Default value is 1000.

    • Service is protected against termination and powering off

    • notifications array[object]

      Service notifications

      Hide notifications attributes Show notifications attributes object

      Service notifications

      • level string Required

        Notification level

        Values are warning or notice.

      • message string Required

        Human notification message

        Minimum length is 1, maximum length is 1024.

      • type string Required

        Notification type

        Values are service_powered_off_removal or service_end_of_life.

      • metadata object Required

        Notification type

    • Kafka Connect configuration values

      Hide kafka-connect-settings attributes Show kafka-connect-settings attributes
      • The total bytes of memory the producer can use to buffer records waiting to be sent to the broker (defaults to 33554432).

        Minimum value is 5242880, maximum value is 134217728.

      • The maximum delay in milliseconds between invocations of poll() when using consumer group management (defaults to 300000).

        Minimum value is 1, maximum value is 2147483647.

      • Specify the default compression type for producers. This configuration accepts the standard compression codecs ('gzip', 'snappy', 'lz4', 'zstd'). It additionally accepts 'none' which is the default and equivalent to no compression.

        Values are gzip, snappy, lz4, zstd, or none.

      • Defines what client configurations can be overridden by the connector. Default is None

        Values are None or All.

      • The interval at which to try committing offsets for tasks (defaults to 60000).

        Minimum value is 1, maximum value is 100000000.

      • The maximum delay that is scheduled in order to wait for the return of one or more departed workers before rebalancing and reassigning their connectors and tasks to the group. During this period the connectors and tasks of the departed workers remain unassigned. Defaults to 5 minutes.

        Minimum value is 0, maximum value is 600000.

      • Records are fetched in batches by the consumer, and if the first record batch in the first non-empty partition of the fetch is larger than this value, the record batch will still be returned to ensure that the consumer can make progress. As such, this is not a absolute maximum.

        Minimum value is 1048576, maximum value is 104857600.

      • Records are fetched in batches by the consumer.If the first record batch in the first non-empty partition of the fetch is larger than this limit, the batch will still be returned to ensure that the consumer can make progress.

        Minimum value is 1048576, maximum value is 104857600.

      • Maximum number of milliseconds to wait for records to flush and partition offset data to be committed to offset storage before cancelling the process and restoring the offset data to be committed in a future attempt (defaults to 5000).

        Minimum value is 1, maximum value is 2147483647.

      • What to do when there is no initial offset in Kafka or if the current offset does not exist any more on the server. Default is earliest

        Values are earliest or latest.

      • This setting will limit the number of record batches the producer will send in a single request to avoid sending huge requests.

        Minimum value is 131072, maximum value is 67108864.

      • This setting gives the upper bound of the batch size to be sent. If there are fewer than this many bytes accumulated for this partition, the producer will 'linger' for the linger.ms time waiting for more records to show up. A batch size of zero will disable batching entirely (defaults to 16384).

        Minimum value is 0, maximum value is 5242880.

      • The timeout in milliseconds used to detect failures when using Kafka’s group management facilities (defaults to 10000).

        Minimum value is 1, maximum value is 2147483647.

      • This setting gives the upper bound on the delay for batching: once there is batch.size worth of records for a partition it will be sent immediately regardless of this setting, however if there are fewer than this many bytes accumulated for this partition the producer will 'linger' for the specified time waiting for more records to show up. Defaults to 0.

        Minimum value is 0, maximum value is 5000.

      • Transaction read isolation level. read_uncommitted is the default, but read_committed can be used if consume-exactly-once behavior is desired.

        Values are read_uncommitted or read_committed.

      • The maximum number of records returned in a single call to poll() (defaults to 500).

        Minimum value is 1, maximum value is 10000.

    • components array[object]

      Service component information objects

      Hide components attributes Show components attributes object
      • component string Required

        Service component name

      • host string Required

        DNS name for connecting to the service component

      • Kafka authentication method. This is a value specific to the 'kafka' service component

        Values are certificate or sasl.

      • port integer(int64) Required

        Port number for connecting to the service component

        Minimum value is 0, maximum value is 65535.

      • route string Required

        Network access route

        Values are dynamic, private, public, or privatelink.

      • usage string Required

        DNS usage name

        Values are primary or replica.

    • Automatic maintenance settings

      Hide maintenance attributes Show maintenance attributes
      • dow string Required

        Day of week for installing updates

        Values are saturday, tuesday, never, wednesday, sunday, friday, monday, or thursday.

      • time string Required

        Time for installing updates, UTC

        Minimum length is 8, maximum length is 8.

      • updates array[object] Required

        Update waiting to be installed

        Hide updates attributes Show updates attributes object

        Update waiting to be installed

        • Description of the update

        • deadline string(date-time)

          Deadline for installing the update

        • start-after string(date-time)

          The earliest time the update will be automatically applied

        • start-at string(date-time)

          The time when the update will be automatically applied

    • Kafka-specific settings

      Hide kafka-settings attributes Show kafka-settings attributes
      • The (optional) comma-delimited setting for the broker to use to verify that the JWT was issued for one of the expected audiences.

        Maximum length is 128. Format should match the following pattern: ^[^\r\n]*$.

      • The maximum allowed session timeout for registered consumers. Longer timeouts give consumers more time to process messages in between heartbeats at the cost of a longer time to detect failures.

        Minimum value is 0, maximum value is 1800000.

      • The number of messages accumulated on a log partition before messages are flushed to disk

        Minimum value is 1, maximum value is 9223372036854775807.

      • OIDC JWKS endpoint URL. By setting this the SASL SSL OAuth2/OIDC authentication is enabled. See also other options for SASL OAuth2/OIDC.

        Maximum length is 2048.

      • The maximum number of connections allowed from each ip address (defaults to 2147483647).

        Minimum value is 256, maximum value is 2147483647.

      • Optional setting for the broker to use to verify that the JWT was created by the expected issuer.

        Maximum length is 128. Format should match the following pattern: ^[^\r\n]*$.

      • The maximum size in bytes of the offset index

        Minimum value is 1048576, maximum value is 104857600.

      • Enable auto creation of topics

      • The interval with which Kafka adds an entry to the offset index

        Minimum value is 0, maximum value is 104857600.

      • The number of bytes of messages to attempt to fetch for each partition (defaults to 1048576). This is not an absolute maximum, if the first record batch in the first non-empty partition of the fetch is larger than this value, the record batch will still be returned to ensure that progress can be made.

        Minimum value is 1048576, maximum value is 104857600.

      • Number of partitions for autocreated topics

        Minimum value is 1, maximum value is 1000.

      • The transaction topic segment bytes should be kept relatively small in order to facilitate faster log compaction and cache loads (defaults to 104857600 (100 mebibytes)).

        Minimum value is 1048576, maximum value is 2147483647.

      • Maximum bytes expected for the entire fetch response (defaults to 10485760). Records are fetched in batches, and if the first record batch in the first non-empty partition of the fetch is larger than this value, the record batch will still be returned to ensure that progress can be made. As such, this is not an absolute maximum.

        Minimum value is 10485760, maximum value is 1048576000.

      • Define whether the timestamp in the message is message create time or log append time.

        Values are CreateTime or LogAppendTime.

      • Idle connections timeout: the server socket processor threads close the connections that idle for longer than this.

        Minimum value is 1000, maximum value is 3600000.

      • The maximum time in ms that a message in any topic is kept in memory before flushed to disk. If not set, the value in log.flush.scheduler.interval.ms is used

        Minimum value is 0, maximum value is 9223372036854775807.

      • Should pre allocate file when create new segment?

      • The amount of time to wait before deleting a file from the filesystem

        Minimum value is 0, maximum value is 3600000.

      • The maximum size of message that the server can receive.

        Minimum value is 0, maximum value is 100001200.

      • The amount of time, in milliseconds, the group coordinator will wait for more consumers to join a new group before performing the first rebalance. A longer delay means potentially fewer rebalances, but increases the time until processing begins. The default value for this is 3 seconds. During development and testing it might be desirable to set this to 0 in order to not delay test execution time.

        Minimum value is 0, maximum value is 300000.

      • The maximum size of local log segments that can grow for a partition before it gets eligible for deletion. If set to -2, the value of log.retention.bytes is used. The effective value should always be less than or equal to log.retention.bytes value.

        Minimum value is -2, maximum value is 9223372036854775807.

      • The maximum jitter to subtract from logRollTimeMillis (in milliseconds). If not set, the value in log.roll.jitter.hours is used

        Minimum value is 0, maximum value is 9223372036854775807.

      • The interval at which to remove transactions that have expired due to transactional.id.expiration.ms passing (defaults to 3600000 (1 hour)).

        Minimum value is 600000, maximum value is 3600000.

      • Enable verification that checks that the partition has been added to the transaction before writing transactional records to the partition

      • Replication factor for autocreated topics

        Minimum value is 1, maximum value is 10.

      • The maximum time before a new log segment is rolled out (in milliseconds).

        Minimum value is 1, maximum value is 9223372036854775807.

      • The purge interval (in number of requests) of the producer request purgatory(defaults to 1000).

        Minimum value is 10, maximum value is 10000.

      • The maximum size of the log before deleting messages

        Minimum value is -1, maximum value is 9223372036854775807.

      • When a producer sets acks to 'all' (or '-1'), min.insync.replicas specifies the minimum number of replicas that must acknowledge a write for the write to be considered successful.

        Minimum value is 1, maximum value is 7.

      • Specify the final compression type for a given topic. This configuration accepts the standard compression codecs ('gzip', 'snappy', 'lz4', 'zstd'). It additionally accepts 'uncompressed' which is equivalent to no compression; and 'producer' which means retain the original compression codec set by the producer.

        Values are gzip, snappy, lz4, zstd, uncompressed, or producer.

      • The maximum difference allowed between the timestamp when a broker receives a message and the timestamp specified in the message

        Minimum value is 0, maximum value is 9223372036854775807.

      • The number of milliseconds to keep the local log segments before it gets eligible for deletion. If set to -2, the value of log.retention.ms is used. The effective value should always be less than or equal to log.retention.ms value.

        Minimum value is -2, maximum value is 9223372036854775807.

      • This configuration controls whether down-conversion of message formats is enabled to satisfy consume requests.

      • Name of the scope from which to extract the subject claim from the JWT. Defaults to sub.

        Maximum length is 128. Format should match the following pattern: ^[^\r\n]*$.

      • The maximum number of incremental fetch sessions that the broker will maintain.

        Minimum value is 1000, maximum value is 10000.

      • The number of hours to keep a log file before deleting it

        Minimum value is -1, maximum value is 2147483647.

      • The minimum allowed session timeout for registered consumers. Longer timeouts give consumers more time to process messages in between heartbeats at the cost of a longer time to detect failures.

        Minimum value is 0, maximum value is 60000.

      • The maximum number of bytes in a socket request (defaults to 104857600).

        Minimum value is 10485760, maximum value is 209715200.

      • The maximum size of a single log file

        Minimum value is 10485760, maximum value is 1073741824.

      • Hide log-cleanup-and-compaction attributes Show log-cleanup-and-compaction attributes
        • How long are delete records retained?

          Minimum value is 0, maximum value is 315569260000.

        • The maximum amount of time message will remain uncompacted. Only applicable for logs that are being compacted

          Minimum value is 30000, maximum value is 9223372036854775807.

        • Controls log compactor frequency. Larger value means more frequent compactions but also more space wasted for logs. Consider setting log.cleaner.max.compaction.lag.ms to enforce compactions sooner, instead of setting a very high value for this option.

          Minimum value is 0.2, maximum value is 0.9.

        • The minimum time a message will remain uncompacted in the log. Only applicable for logs that are being compacted.

          Minimum value is 0, maximum value is 9223372036854775807.

        • The default cleanup policy for segments beyond the retention window

          Values are delete, compact, or compact,delete.

      • Log retention window in minutes for offsets topic

        Minimum value is 1, maximum value is 2147483647.

      • The number of milliseconds to keep a log file before deleting it (in milliseconds), If not set, the value in log.retention.minutes is used. If set to -1, no time limit is applied.

        Minimum value is -1, maximum value is 9223372036854775807.

    • disk-size integer(int64)

      TODO UNIT disk space for data storage

      Minimum value is 0.

    • node-memory integer(int64)

      TODO UNIT of memory for each node

      Minimum value is 0.

    • uri string

      URI for connecting to the service (may be absent)

    • service_uri parameterized into key-value pairs

    • Whether Schema-Registry is enabled

    • version string

      Kafka version

    • created-at string(date-time)

      Service creation timestamp (ISO 8601)

    • plan string Required

      Subscription plan

    • users array[object]

      List of service users

      Hide users attributes Show users attributes object
GET /dbaas-kafka/{name}
curl \
 -X GET https://api-ch-gva-2.exoscale.com/v2/dbaas-kafka/{name}
Response examples (200)
{
  "updated-at": "2024-05-04T09:42:00+00:00",
  "authentication-methods": {
    "certificate": true,
    "sasl": true
  },
  "node-count": 42,
  "connection-info": {
    "nodes": [
      "string"
    ],
    "access-cert": "string",
    "access-key": "string",
    "connect-uri": "string",
    "rest-uri": "string",
    "registry-uri": "string"
  },
  "node-cpu-count": 42,
  "kafka-rest-enabled": true,
  "integrations": [
    {
      "description": "string",
      "settings": {},
      "type": "string",
      "is-enabled": true,
      "source": "string",
      "is-active": true,
      "status": "string",
      "id": "string",
      "dest": "string"
    }
  ],
  "zone": "string",
  "node-states": [
    {
      "name": "string",
      "progress-updates": [
        {
          "completed": true,
          "current": 42,
          "max": 42,
          "min": 42,
          "phase": "stream",
          "unit": "string"
        }
      ],
      "role": "standby",
      "state": "leaving"
    }
  ],
  "name": "string",
  "kafka-connect-enabled": true,
  "type": "string",
  "state": "running",
  "ip-filter": [
    "string"
  ],
  "schema-registry-settings": {
    "leader_eligibility": true,
    "topic_name": "_schemas"
  },
  "backups": [
    {
      "backup-name": "string",
      "backup-time": "2024-05-04T09:42:00+00:00",
      "data-size": 42
    }
  ],
  "kafka-rest-settings": {
    "producer_compression_type": "gzip",
    "name_strategy_validation": true,
    "name_strategy": "topic_name",
    "consumer_enable_auto_commit": true,
    "producer_acks": "1",
    "consumer_request_max_bytes": 67108864,
    "producer_max_request_size": 1048576,
    "simpleconsumer_pool_size_max": 25,
    "producer_linger_ms": 0,
    "consumer_request_timeout_ms": 1000
  },
  "termination-protection": true,
  "notifications": [
    {
      "level": "warning",
      "message": "string",
      "type": "service_powered_off_removal",
      "metadata": {}
    }
  ],
  "kafka-connect-settings": {
    "producer_buffer_memory": 8388608,
    "consumer_max_poll_interval_ms": 300000,
    "producer_compression_type": "gzip",
    "connector_client_config_override_policy": "None",
    "offset_flush_interval_ms": 60000,
    "scheduled_rebalance_max_delay_ms": 300000,
    "consumer_fetch_max_bytes": 52428800,
    "consumer_max_partition_fetch_bytes": 1048576,
    "offset_flush_timeout_ms": 5000,
    "consumer_auto_offset_reset": "earliest",
    "producer_max_request_size": 1048576,
    "producer_batch_size": 1024,
    "session_timeout_ms": 10000,
    "producer_linger_ms": 100,
    "consumer_isolation_level": "read_uncommitted",
    "consumer_max_poll_records": 500
  },
  "components": [
    {
      "component": "string",
      "host": "string",
      "kafka-authentication-method": "certificate",
      "port": 42,
      "route": "dynamic",
      "usage": "primary"
    }
  ],
  "maintenance": {
    "dow": "saturday",
    "time": "string",
    "updates": [
      {
        "description": "string",
        "deadline": "2024-05-04T09:42:00+00:00",
        "start-after": "2024-05-04T09:42:00+00:00",
        "start-at": "2024-05-04T09:42:00+00:00"
      }
    ]
  },
  "kafka-settings": {
    "sasl_oauthbearer_expected_audience": "string",
    "group_max_session_timeout_ms": 1800000,
    "log_flush_interval_messages": 9223372036854775807,
    "sasl_oauthbearer_jwks_endpoint_url": "string",
    "max_connections_per_ip": 42,
    "sasl_oauthbearer_expected_issuer": "string",
    "log_index_size_max_bytes": 10485760,
    "auto_create_topics_enable": true,
    "log_index_interval_bytes": 4096,
    "replica_fetch_max_bytes": 42,
    "num_partitions": 42,
    "transaction_state_log_segment_bytes": 104857600,
    "replica_fetch_response_max_bytes": 42,
    "log_message_timestamp_type": "CreateTime",
    "connections_max_idle_ms": 540000,
    "log_flush_interval_ms": 42,
    "log_preallocate": false,
    "log_segment_delete_delay_ms": 60000,
    "message_max_bytes": 1048588,
    "group_initial_rebalance_delay_ms": 3000,
    "log_local_retention_bytes": 42,
    "log_roll_jitter_ms": 42,
    "transaction_remove_expired_transaction_cleanup_interval_ms": 3600000,
    "transaction_partition_verification_enable": true,
    "default_replication_factor": 42,
    "log_roll_ms": 42,
    "producer_purgatory_purge_interval_requests": 42,
    "log_retention_bytes": 42,
    "min_insync_replicas": 1,
    "compression_type": "gzip",
    "log_message_timestamp_difference_max_ms": 42,
    "log_local_retention_ms": 42,
    "log_message_downconversion_enable": true,
    "sasl_oauthbearer_sub_claim_name": "string",
    "max_incremental_fetch_session_cache_slots": 1000,
    "log_retention_hours": 42,
    "group_min_session_timeout_ms": 6000,
    "socket_request_max_bytes": 42,
    "log_segment_bytes": 42,
    "log-cleanup-and-compaction": {
      "log_cleaner_delete_retention_ms": 86400000,
      "log_cleaner_max_compaction_lag_ms": 42,
      "log_cleaner_min_cleanable_ratio": 0.5,
      "log_cleaner_min_compaction_lag_ms": 42,
      "log_cleanup_policy": "delete"
    },
    "offsets_retention_minutes": 10080,
    "log_retention_ms": 42
  },
  "disk-size": 42,
  "node-memory": 42,
  "uri": "string",
  "uri-params": {},
  "schema-registry-enabled": true,
  "version": "string",
  "created-at": "2024-05-04T09:42:00+00:00",
  "plan": "string",
  "users": [
    {
      "type": "string",
      "username": "string",
      "password": "string",
      "access-cert": "string",
      "access-cert-expiry": "2024-05-04T09:42:00+00:00",
      "access-key": "string"
    }
  ]
}