[Original] (d) Sparse Linux memory model of the Memory Model


    Read the fucking source code! –By Lu Xun

    A picture is worth a thousand words. –By Golgi


    Kernel Version: 4.14

    ARM64 processor, Contex-A53, binuclear

    Use tools: Source Insight 3.5, Visio

1 Introduction

Down the previous analysis, we came to the bootmem_init () function, and the thought that an article can get, probably after the sweep again the code, I silently break it into two parts.
    bootmem_init () function code is as follows:

void __init bootmem_init(void)
    unsigned long min, max;

    min = PFN_UP(memblock_start_of_DRAM());
    max = PFN_DOWN(memblock_end_of_DRAM());

    early_memtest(min << PAGE_SHIFT, max << PAGE_SHIFT);

    max_pfn = max_low_pfn = max;

     * Sparsemem tries to allocate bootmem in memory_present(), so must be
     * done after the fixed reservations.

    zone_sizes_init(min, max);


This section, we will look Sparse Memory Model.
    Speaking before the Linux memory model, knowledge need to add two points: PFN and NUMA.

1.1 physical frame number(PFN)

Earlier we had about the virtual addresses to physical address mapping process, and system memory management is based on page units:
    page: linear addresses are divided into groups in units of fixed length, called pages, such as the typical size of 4K, continuous linear internal page address is mapped into consecutive physical addresses;
    page frame: memory storage region is divided into a fixed length, called pages block, also called a physical page. Each page frame contains a page, and a page length is the same page frame, the kernel used to struct page associated with physical pages.


As __page_to_pfn This depends upon the particular model of the physical memory, as will be described.

1.2 NUMA

  • UMA: Uniform Memory Access,所有处理器对内存的访问都是一致的:

As it can be seen from the figure above, when the processor and Core increases when the memory bandwidth will become a bottleneck.

  • NUMA: Non Uniform Memory Access,非一致性内存访问:

As it can be seen from the figure, each CPU to access local memory, faster, less delay. Of course, the overall configuration of a memory pool memory, the CPU can access remote memory, relatively slower, larger delay. Current understanding of the NUMA-limited, will meet in the kernel code related, probably know what category it belongs to.

2. Linux memory model


General processor architecture supports one or more memory model, this has been determined at compile time, Sparse Memory Model such as the current in the ARM64 in use.

  • Flat Memory
                Continuous physical memory address, this is the Linux memory model originally used. When the memory is empty also can use this model, just struct page * size was positively correlated with the physical address mem_map array memory is empty wasteful.

  • Discontiguous Memory
                The presence of voids physical memory, as proposed Sparse Memory This memory model is gradually being abandoned.

  • Sparse Memory
                There is a physical memory holes, and supports hot-plug memory, in units of the management section, which is analyzed below.

Under Linux of the three memory models, struct page to a physical page frame mapping methods are not the same, you can view specific __pfn_to_page / __ page_to_pfn defined in include / asm-generic / memory_model.h file.

About memory model, refer to Memory: the flat, the discontiguous, and the sparse

3. Sparse Memory

This section analyzes the model is Sparse Memory of ARM64, UMA (does not support the ARM NUMA linux4.14 in).

3.1 mem_section

Sparse Memory模型中,section是管理内存online/offline的最小内存单元,在ARM64中,section的大小为1G,而在Linux内核中,通过一个全局的二维数组struct mem_section **mem_section来维护映射关系。


It is known that a pfn, can eventually find the corresponding struct page structure by __pfn_to_section (pfn).

3.2 sparse_init


In this function, the first distribution userMap, this userMap associated with memory recall mechanism, will be described with bitmap 4bit page block (a pageblock size is usually a power of two, such MAX_ORDER-1) type of migration:

/* Bit indices that affect a whole block of pages */
enum pageblock_bits {
    PB_migrate_end = PB_migrate + 3 - 1,
            /* 3 bits required for migrate types */
    PB_migrate_skip,/* If set the block is skipped by compaction */

     * Assume the bits will always align on a word. If this assumption
     * changes then get/set pageblock needs updating.

sparse memory模型会为每一个section都分配一个usermap,最终的物理页面的压缩,迁移等操作,都跟这些位相关,如下图所示:

sparse_init函数中,另一部分的作用是遍历所有present section,然后将其映射到vmemmap区域空间。vmemmap区域空间,在之前的文章中也提到过。执行完后,整体的效果如下图所示:

About Sparse Memory Model so much on the first analysis, the specific modules only in conjunction with sparse memory, understanding will be smoother.

Believe it is easy buckle details, and once into the details, it is easy to become core devil, too difficult.

Leave a Reply