GET _cluster/settings?flat_settings=true&include_defaults=false
we want one shard per node, always
"cluster.routing.allocation.balance.shard": 0.01, "cluster.routing.allocation.balance.index": 99, "cluster.routing.allocation.balance.threshold": 0.01, "cluster.routing.allocation.balance.prefer_primary": true
although we keep the disk allocation decider enabled
"cluster.routing.allocation.disk.threshold_enabled": true, "cluster.routing.allocation.disk.watermark.low": "90%", "cluster.routing.allocation.disk.watermark.high": "93%", "cluster.routing.allocation.disk.watermark.flood_stage": "96%"
reduce shards cap per node by half – one should have large shards (10-50gb and up to 200 million docs) hence a reduced amount of those
PUT _cluster/settings { "persistent":{ "cluster.max_shards_per_node": 500 } }
assuming 8 pri shards and 1 replica, one should not need more than 16 shards for queries – and as we use pri shard size only policies, let’s assume no more than 4 indices – that is fine ONLY IF YOU HAVE SPECIFIC INDEX PATTERNS (if you have global index patterns walking across many indices, that won’t do)
echo $(( 16 * 4 )) PUT /_cluster/settings { "transient": { "action.search.shard_count.limit": 64 } }
query least busy replicas
PUT /_cluster/settings { "persistent": { "cluster.routing.use_adaptive_replica_selection": true } }
eventually enable that fork-feature and check – by “fork” I mean that’s opensearch only, apparently.
PUT _cluster/settings { "persistent": { "shard_indexing_pressure": { "enabled": true, "enforced": false } } } GET /_nodes/_local/stats/shard_indexing_pressure
curl -sk https://localhost:9200/_cluster/settings -u admin:PASSWORD | jq cat <<EOF | curl -sk -X PUT -H "Content-Type: application/json" \ "https://localhost:9200/_cluster/settings?flat_settings=true" -u admin:PASSWORD \ -d @- | jq { "persistent" : { "plugins.index_state_management.job_interval" : "1" } }, { "transient" : { "plugins.index_state_management.job_interval" : "1" } } EOF
https://www.elastic.co/guide/en/elasticsearch/reference/7.17/cluster-update-settings.html
https://opensearch.org/docs/latest/im-plugin/ism/settings/
https://opensearch.org/docs/latest/install-and-configure/configuring-opensearch/index-settings/
https://opensearch.org/docs/latest/tuning-your-cluster/performance/