[okl4-developer] OKL4 v2.1: About memory layout
Frank Kaiser
frank.kaiser at opensynergy.com
Wed Sep 24 01:14:46 EST 2008
Hello
Further testing showed that manipulating the parameter heap_base is somewhat critical: it seems to render the image defective in some cases for whatever reason. The ELF Weaver patches information about the image's properties into the data structure init_struct, and this may become inconsistent (just a guess).
But changing the heap size seems to be less critical, but has an interesting impact on the segment allocation. Here an example, when setting the heap size to 0x38000:
Program Headers:
Type Offset VirtAddr PhysAddr FileSiz MemSiz Flg Align
LOAD 0x008000 0xf0000000 0x20000000 0x2022c 0x2390c RWE 0x8000
LOAD 0x02c000 0xf0024000 0x20024000 0x06000 0x06000 RW 0x8000
LOAD 0x038000 0x90000000 0x200c8000 0x0dfaf 0x0dfaf R E 0x8000
LOAD 0x046000 0x90016000 0x2002b000 0x00234 0x00364 RW 0x8000
LOAD 0x048000 0x90100000 0x20088000 0x0526f 0x0526f R E 0x8000
LOAD 0x04d270 0x9010d270 0x2002f270 0x00118 0x00238 RW 0x8000
LOAD 0x050000 0x90200000 0x200d8000 0x055ef 0x055ef R E 0x8000
LOAD 0x0555f0 0x9020d5f0 0x200d75f0 0x00148 0x00518 RW 0x8000
LOAD 0x058000 0x90300000 0x20114000 0x05df8 0x05df8 R E 0x8000
LOAD 0x05e000 0x9030e000 0x20111000 0x00254 0x00440 RW 0x8000
LOAD 0x060000 0x90400000 0x20130000 0x0598b 0x0598b R E 0x8000
LOAD 0x06598c 0x9040d98c 0x2011b98c 0x00130 0x00220 RW 0x8000
LOAD 0x066000 0x90056000 0x20136000 0x03000 0x03000 RW 0x1000
Surprise: the 4 MB hole has disappeared! In general, all segments carrying .text and .rodata sections and some of the segments carrying .data and .bss sections have become new offsets, although the sizes did not change (the email below does not show the latest state of VSPI and SPI_TEST). Here another example with heap size = 0x48000:
Program Headers:
Type Offset VirtAddr PhysAddr FileSiz MemSiz Flg Align
LOAD 0x008000 0xf0000000 0x20000000 0x2022c 0x2390c RWE 0x8000
LOAD 0x02c000 0xf0024000 0x20024000 0x06000 0x06000 RW 0x8000
LOAD 0x038000 0x90000000 0x200d8000 0x0dfaf 0x0dfaf R E 0x8000
LOAD 0x046000 0x90016000 0x2002b000 0x00234 0x00364 RW 0x8000
LOAD 0x048000 0x90100000 0x20088000 0x0526f 0x0526f R E 0x8000
LOAD 0x04d270 0x9010d270 0x2002f270 0x00118 0x00238 RW 0x8000
LOAD 0x050000 0x90200000 0x200e8000 0x055ef 0x055ef R E 0x8000
LOAD 0x0555f0 0x9020d5f0 0x200e75f0 0x00148 0x00518 RW 0x8000
LOAD 0x058000 0x90300000 0x20124000 0x05df8 0x05df8 R E 0x8000
LOAD 0x05e000 0x9030e000 0x20121000 0x00254 0x00440 RW 0x8000
LOAD 0x060000 0x90400000 0x20140000 0x0598b 0x0598b R E 0x8000
LOAD 0x06598c 0x9040d98c 0x2012b98c 0x00130 0x00220 RW 0x8000
LOAD 0x066000 0x90056000 0x20146000 0x03000 0x03000 RW 0x1000
The segment offsets have changed again, so that the overall image size has grown by 64 kB, although the segment sizes are the same. Finally an attempt with heap size = 0x3C000:
Program Headers:
Type Offset VirtAddr PhysAddr FileSiz MemSiz Flg Align
LOAD 0x008000 0xf0000000 0x20000000 0x2022c 0x2390c RWE 0x8000
LOAD 0x02c000 0xf0024000 0x20024000 0x06000 0x06000 RW 0x8000
LOAD 0x038000 0x90000000 0x200d0000 0x0dfaf 0x0dfaf R E 0x8000
LOAD 0x046000 0x90016000 0x2002b000 0x00234 0x00364 RW 0x8000
LOAD 0x048000 0x90100000 0x20088000 0x0526f 0x0526f R E 0x8000
LOAD 0x04d270 0x9010d270 0x2002f270 0x00118 0x00238 RW 0x8000
LOAD 0x050000 0x90200000 0x20100000 0x055ef 0x055ef R E 0x8000
LOAD 0x0555f0 0x9020d5f0 0x200cd5f0 0x00148 0x00518 RW 0x8000
LOAD 0x058000 0x90300000 0x20108000 0x05df8 0x05df8 R E 0x8000
LOAD 0x05e000 0x9030e000 0x200df000 0x00254 0x00440 RW 0x8000
LOAD 0x060000 0x90400000 0x20130000 0x0598b 0x0598b R E 0x8000
LOAD 0x06598c 0x9040d98c 0x2010f98c 0x00130 0x00220 RW 0x8000
LOAD 0x066000 0x90056000 0x20138000 0x03000 0x03000 RW 0x1000
Segment offsets are different again, and compared to the 'heap size = 0x38000' the image is 8 kB bigger.
The conclusion is that the memory allocation algorithm carried out be the GNU loader under control of the ELF Adorn and ELF Weaver tools is somewhat unpredictable, and I do not see why the manipulation of the heap size has such a significant effect. As long as the heap size is not a multiple of 1M, the image is compact, i. e does not show the 4 MB hole. I do not believe that this behaviour is intentional; it looks more likely that there is a bug in the tool chain responsible for the image allocation. Who can shed light on this?
I forgot the reason behind this research: we had to make the binary image fit into a 2 MB NOR flash, and this was difficult with the original image layout.
Regards
Frank
From: developer-bounces at okl4.org [mailto:developer-bounces at okl4.org] On Behalf Of Frank Kaiser
Sent: Friday, September 19, 2008 4:22 PM
To: developer at okl4.org
Subject: [okl4-developer] OKL4 v2.1: About memory layout
Hello
Since the U-Boot of my ATMEL ARM platform is not able to load ELF images, I have to convert them to binaries. This is automatically done when specifying 'boot_binary = True' in the machine's class definition. With additional drivers the Pistachio/Iguana ELF image bypasses the size of 1 MB, and close to that point the size of of the binary jumps to more than 5 MB. This was a surprise and a shortcoming, since I have an unreliable Ethernet interface on one hardware. Downloading >5 MB thru a serial line with 115 kbit/s takes >15 minutes which I cannot afford. So I started looking for the reason of the image increase.
Readelf showed a gap of 4 MB in the ELF image like this:
Program Headers:
Type Offset VirtAddr PhysAddr FileSiz MemSiz Flg Align
LOAD 0x008000 0xf0000000 0x20000000 0x2022c 0x2390c RWE 0x8000
LOAD 0x02c000 0xf0024000 0x20024000 0x06000 0x06000 RW 0x8000
LOAD 0x038000 0x90000000 0x20088000 0x0dfaf 0x0dfaf R E 0x8000
LOAD 0x046000 0x90016000 0x2002b000 0x00234 0x00364 RW 0x8000
LOAD 0x048000 0x90100000 0x20098000 0x0526f 0x0526f R E 0x8000
LOAD 0x04d270 0x9010d270 0x2002f270 0x00118 0x00238 RW 0x8000
LOAD 0x050000 0x90200000 0x200c0000 0x0559f 0x0559f R E 0x8000
LOAD 0x0555a0 0x9020d5a0 0x2009f5a0 0x00148 0x00518 RW 0x8000
LOAD 0x058000 0x90300000 0x200e0000 0x06313 0x06313 R E 0x8000
LOAD 0x05e314 0x9030e314 0x200c9314 0x00244 0x00430 RW 0x8000
LOAD 0x060000 0x90400000 0x200e8000 0x0529f 0x0529f R E 0x8000
LOAD 0x0652a0 0x9040d2a0 0x200cd2a0 0x00120 0x00210 RW 0x8000
LOAD 0x066000 0x90056000 0x20510000 0x03000 0x03000 RW 0x1000
Section to Segment mapping:
Segment Sections...
00 kernel.text kernel.rodata kernel.kdebug kernel.init kernel.roinit kernel.data kernel.got kernel.got.plt kernel.kdebug-data kernel.bss
01 kernel.kspace kernel.traps kernel.utcb_page
02 ig_server.text ig_server.rodata
03 ig_server.data ig_server.got ig_server.bss
04 event.text event.rodata
05 event.data event.got event.bss
06 vostimer.text vostimer.rodata
07 vostimer.data vostimer.got vostimer.bss
08 vspi.text vspi.rodata
09 vspi.data vspi.got vspi.bss
10 spi_test.text spi_test.rodata
11 spi_test.data spi_test.got spi_test.bss
12 bootinfo
Checking the page tables with the kernel debugger shows that 4 memory block à 1 MB between 0x20100000 and 0x20500000 are allocated by the kernel. Obviously is this the kernel heap.
In section 4.4.3 of the ELF Weaver Manual it is explained that by default the heap will have a size of 4 MB and that it will be allocated close to the kernel. The above layout shows that the heap is placed in the next 1 MB segment after the kernel sections. The outcome of this placement is that a binary created from this segment allocation must contain the 4 MB kernel heap, because the bootinfo section is located beyond.
The placement of the kernel heap, as outlined in the manual, is motivated by addressing constraints, for instance, of the ARM architecture. However, such constraints only apply to virtual addresses, which are seen by the CPU core, not the physical ones, which are seen by the MMU. For fulfilling an addressing constraint it should be sufficient to set the heap's virtual address close to the kernel's virtual address, so the heap could be placed at the end of the physically allocated memory, thus keeping the binary image small.
By default the build system does not write heap parameters to the file weaver.xml, although doing so is supported. In tools/build.py there is the class WeaverKernel which can take the parameter heap on instantiation to set the kernel heap size. The class' method element() recognizes the attribute heap_base to set the heap's physical start address. The class WeaverKernel is used in the kernel's SConscript file. Adding "heap = ..." to the call results in setting the heap size in weaver.xml. With a subsequent "weaver.heap_base = ..." the heap start address can be set.
The problem is to determine a sufficient value for the start address, since when parsing the kernel's SConscript file, the physical size of the image is not yet known. It is wishful to derive the start address from the overall image size including the bootinfo section. It should be possible to calculate that in the main SConstruct file after all SConscript files have been parsed, but I have not yet found out how to do this.
While playing with the heap attributes I found that the ELF Weaver generates an error message when the heap's distance to the kernel code is bigger than 64M. Based on my assumption that the physical addresses are not relevant, I wonder why this is the case.
Regards
Frank
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.okl4.org/pipermail/developer/attachments/20080923/81f6a121/attachment-0001.htm
More information about the Developer
mailing list