elastic/osearch cluster tuning

overall status & checks

GET _cluster/settings?flat_settings=true&include_defaults=false
GET _cluster/settings?flat_settings=true
GET /_nodes/_local/stats/shard_indexing_pressure

cluster optimization

we want one shard per node, whatever happens - this is what a cluster and replicas are for

PUT _cluster/settings
{
  "persistent":{
    "cluster.routing.allocation.balance.shard": 0.01,
    "cluster.routing.allocation.balance.index": 99,
    "cluster.routing.allocation.balance.threshold": 0.01,
    "cluster.routing.allocation.balance.prefer_primary": true
  }
}

write optimization

one should have large shards (10-50gb and up to 200 million docs) hence a reduced amount of shards

default is 1000 shards per node – so you could easily reduce that amount by half

PUT _cluster/settings
{
   "persistent":{
      "cluster.max_shards_per_node": 500
   }
}

read optimization

use per-ds index patterns (instead of setting action.search.shard_count.limit)

query least busy replicas

PUT /_cluster/settings
{
  "persistent": {
    "cluster.routing.use_adaptive_replica_selection": true
  }
}

eventually enable that fork-feature and check – by “fork” I mean that’s opensearch only, apparently.

PUT _cluster/settings
{
  "persistent": {
    "shard_indexing_pressure": {
      "enabled": true,
      "enforced": false
    }
  }
}

resources

cluster settings

https://www.elastic.co/guide/en/elasticsearch/reference/7.17/cluster-update-settings.html

index settings

https://opensearch.org/docs/latest/install-and-configure/configuring-opensearch/index-settings/

performance

https://opensearch.org/docs/latest/tuning-your-cluster/performance/