Understanding how to index and query data is fundamental to working with Elasticsearch. In this article, we’ll explore how to create indices, add documents, and perform various types of searches using Elasticsearch’s REST API.

Introduction

After setting up Elasticsearch, the next step is to learn how to store and retrieve data. Elasticsearch’s REST API provides a simple yet powerful way to interact with your data. We’ll cover the basics of indexing documents and performing different types of queries.

Basic Concepts

Documents

Documents are JSON objects that contain your data. Each document belongs to a type and resides in an index.

Indices

An index is a collection of documents that have similar characteristics. Think of it as a database in traditional SQL terms.

Mapping

Mapping defines how a document and its fields are stored and indexed. Elasticsearch can automatically detect field types, but you can also define them explicitly.

Creating an Index

Basic Index Creation

PUT /products
{
  "settings": {
    "number_of_shards": 1,
    "number_of_replicas": 1
  },
  "mappings": {
    "properties": {
      "name": { "type": "text" },
      "price": { "type": "float" },
      "category": { "type": "keyword" },
      "description": { "type": "text" },
      "in_stock": { "type": "boolean" }
    }
  }
}

Index Settings

  • number_of_shards: Number of primary shards
  • number_of_replicas: Number of replica shards
  • refresh_interval: How often to refresh the index

Indexing Documents

Single Document Indexing

POST /products/_doc/1
{
  "name": "Elasticsearch Guide",
  "price": 49.99,
  "category": "books",
  "description": "A comprehensive guide to Elasticsearch",
  "in_stock": true
}

Bulk Indexing

POST /products/_bulk
{ "index": { "_id": "2" } }
{ "name": "Kibana Dashboard", "price": 29.99, "category": "software", "description": "Visualization tool for Elasticsearch", "in_stock": true }
{ "index": { "_id": "3" } }
{ "name": "Logstash Pipeline", "price": 39.99, "category": "software", "description": "Data processing pipeline", "in_stock": false }

Basic Queries

Match Query

Searches for documents that match a specific text:

GET /products/_search
{
  "query": {
    "match": {
      "description": "guide"
    }
  }
}

Term Query

Searches for exact matches:

GET /products/_search
{
  "query": {
    "term": {
      "category": "books"
    }
  }
}

Range Query

Searches for values within a range:

GET /products/_search
{
  "query": {
    "range": {
      "price": {
        "gte": 20,
        "lte": 50
      }
    }
  }
}

Combining Queries

Bool Query

Combines multiple queries using boolean logic:

GET /products/_search
{
  "query": {
    "bool": {
      "must": [
        { "match": { "description": "tool" } }
      ],
      "filter": [
        { "term": { "in_stock": true } },
        { "range": { "price": { "lte": 50 } } }
      ]
    }
  }
}

Query Results

Understanding the Response

A typical search response includes:

  • took: Time taken to execute the query
  • timed_out: Whether the query timed out
  • _shards: Information about shards
  • hits: Matching documents and their scores

Pagination

Use from and size for pagination:

GET /products/_search
{
  "from": 0,
  "size": 10,
  "query": {
    "match_all": {}
  }
}

Best Practices

  1. Index Design

    • Choose appropriate field types
    • Use analyzers wisely
    • Consider index patterns

  2. Query Optimization

    • Use filters for exact matches
    • Combine queries efficiently
    • Monitor query performance

  3. Data Management

    • Use bulk operations for large datasets
    • Implement proper error handling
    • Monitor index health

Common Issues and Solutions

Mapping Conflicts

  • Define explicit mappings
  • Use index templates
  • Handle dynamic mapping carefully

Performance Issues

  • Optimize shard settings
  • Use appropriate query types
  • Monitor resource usage

Next Steps

Now that you understand basic indexing and querying, you can:

  1. Explore advanced query types
  2. Learn about aggregations
  3. Implement search highlighting
  4. Set up proper security

In the next article, we’ll dive into advanced querying techniques and aggregations.

Conclusion

Mastering indexing and querying is crucial for effective Elasticsearch usage. By understanding how to:

  • Create and manage indices
  • Index documents efficiently
  • Perform various types of queries
  • Combine queries for complex searches

You’ll be well-equipped to build powerful search applications. Remember to:

  • Follow best practices for index design
  • Optimize your queries
  • Monitor performance
  • Handle errors gracefully

Stay tuned for the next article where we’ll explore advanced querying techniques and aggregations in detail.