Doris Memory Insufficiency Challenges and Solutions

Cloud & DevOps Hub 0 554

Apache Doris, as a high-performance real-time analytical database, occasionally encounters memory insufficiency issues during intensive computation tasks. This article explores practical strategies to address memory bottlenecks while maintaining query efficiency, with technical insights suitable for engineers and architects.

Doris Memory Insufficiency Challenges and Solutions

When handling complex queries involving large-scale joins, aggregations, or sorting operations, Doris may throw memory allocation errors resembling "Memory limit exceeded" or "Allocator failed allocation." These errors typically occur when:

  1. Concurrent queries exceed allocated memory resources
  2. Single query requires more memory than configured limits
  3. Data skew causes uneven memory distribution across nodes

To diagnose memory issues effectively, administrators can utilize Doris's built-in monitoring tools:

SHOW BACKENDS\G;  -- Check node memory usage
SHOW PROC "/current_queries";  -- Monitor running queries

This helps identify whether memory pressure stems from query patterns, configuration settings, or hardware limitations.

Configuration Adjustments
Modify be.conf parameters for Backend nodes:

mem_limit=80%  # Set to 80% of physical RAM
storage_page_cache_limit=40%  # Adjust cache allocation

Set Frontend parameters in fe.conf:

query_mem_limit=8589934592  # 8GB per query limit

Query Optimization Techniques

  1. Partition Pruning:
    Leverage partitioned tables to reduce scanned data: 
    SELECT * FROM sales WHERE dt BETWEEN '2023-01-01' AND '2023-01-31';
  2. Materialized Views:
    Pre-aggregate frequently used dimensions: 
    CREATE MATERIALIZED VIEW store_sales_mv AS  
SELECT store_id, SUM(amount) FROM sales GROUP BY store_id;
  3. Batch Processing:
    Split large ETL jobs into smaller batches using LIMIT/OFFSET: 
    SELECT * FROM orders ORDER BY id LIMIT 10000 OFFSET 0;

Resource Management Strategies
Implement workload isolation through resource groups:

CREATE RESOURCE GROUP etl_group  
TO  
   (user='etl_user', role='etl')  
WITH  
   'cpu_share=400', 'mem_limit=50%';

This prevents analytical queries from interfering with ETL processes.

Hardware Considerations
For production environments handling TB-scale data:

  • Maintain 1:4 ratio between Doris Backend memory and storage capacity
  • Allocate separate disks for data storage and scratch space
  • Enable swap space as emergency buffer (minimum 32GB)

Case Study: E-commerce Platform Optimization
A major retailer reduced memory errors by 78% through:

  • Enabling query queueing (enable_query_queue=true)
  • Converting 12 frequent join queries to materialized views
  • Implementing tiered storage for historical data

While memory tuning remains critical, engineers should balance optimizations with cluster expansion needs. For persistent memory issues exceeding 30% of operational time, consider horizontal scaling by adding Backend nodes rather than over-optimizing configurations.

Regular maintenance routines including histogram analysis of query memory usage and periodic review of data distribution patterns help sustain system health. Doris's evolving memory management features, like spill-to-disk in version 2.0+, provide additional safeguards against out-of-memory incidents.

Related Recommendations：