Assignment 4 - 2021

 

Team 1: page fault on calloc 

Using phase 4 of Shibata's team, write a program that continually gets memory (calloc) and writes on a structure, creating a circular linked list (where each cell is in this structure), i.e., a cell (this structure) must point to the former one and to the next one. Make sure you write on the data structure in order to force a major page fault. Explain how virtual memory was allocated by /proc/$PID/maps (the process may be blocked by a getchar). Did the calloc took memory from heap or from stack? After some time, due to virtual memory, data must go to disk in order to free memory to more data through do_swap_page. How much memory did your process get? Make sure the swap area is very large, greater than 4GB. You may experiment getting little or big chunks of memory using calloc. Does it change how much memory your process get?

Team 2: shared memory

Using phase 4 of Shibata's team, implement a program that creates a shared memory greater than the physical memory. One process must continually and sequentially write on this shared memory and another, must read.  A page swap must happen after some writes due to the size of the shared memory. Insert a breakpoint to study the page fault and the swap. Explain. Obs: observe the effect of different sizes of the physical memory in qemu. Obs: observe the effect of different size of the physical memory in qemu. Explain how virtual memory was allocated by /proc/$PID/maps  (the process may be blocked by a getchar).

Team 3: fork bomb

A fork bomb is a program that continually forks. Ex: while(1) {fork();};
Using phase 4 of Shibata's team, implement a fork bomb. You may get errors due to a run out of memory in the kernel or in the user space. Explain what error did you get. It is very likely that you got a run out of memory in the kernel. If so, modify the fork bomb in order to allocate memory before each fork (you may declare data as global data taking a lot of memory, i.e., make sure you use this "mallocated" memory), so that, the user memory runs out before the kernel memory. In this case, you may get page faults that will swap data between memory and disk until a future fatal error happens. Study it.  Let the program run freely. What error do you get? Insert a breakpoint closer to the interrupt vector when this error happens. Explain. Obs: observe the effect of differente sizes of the physical memory in qemu. Explain how virtual memory was allocated by /proc/$PID/maps (the process may be blocked by a getchar).

Team 4: page table: physical address

The Linux kernel provides information about a page table of any process $pid in /proc/$pid/pagemap . The structure of this file is detailed in https://www.kernel.org/doc/Documentation/vm/pagemap.txt

This informatiion was used to create the program "pagemap" that runs in a normal Linux in https://github.com/dwks/pagemap . It displays a process page table in a friendly way.

Adapt this code to our environment (versatile board emulated by qemu).

Create the program "large" that consumes memory by, for instance, using a large vector. Monitor its memory usage by "pagemap". 

Create a program that combines the results of /proc/$pid/maps (shows the location of virtual memory) with /proc/$pid/pagemap (shows the physical memory). 

Team 5: stack, recursion

Using phase 4 of Shibata's team, write the recursive function "recursive(x1,x2,x3)" which increments x1, x2, x3 and calls itself. The stack must grow until occurs a page_fault. Explain how does Linux treat this page_fault. Let the program run freely. What error do you get? Insert a breakpoint closer to the interrupt vector when this error happens. Explain. Obs: observe the effect of differente sizes of the physical memory in qemu. Explain how virtual memory was allocated by /proc/$PID/maps (the process may be blocked by a getchar). Do this recursive program uses virtual memory swapping pages to the swap area?

Team 6: swapping

Using phase 4 of Shibata's team, implement the program "large" that uses 1/3 of physical memory. This program may declare a large global vector. For each vector position, a number is written. You may execute four instances of "large" in background (in a shell: # large&). Insert breakpoints to study how the swap is performed.  Let the program run freely. Do you get any error?  Insert a breakpoint closer to the interrupt vector when this error happens. Explain. Obs: observe the effect of differente sizes of the physical memory in qemu. Explain how virtual memory was allocated by /proc/$PID/maps (each process may be blocked by a getchar).

Team 7: /proc/$PID/pagemap

Read in:
https://www.kernel.org/doc/Documentation/vm/pagemap.txt

Since Linux 4.0 only users with the CAP_SYS_ADMIN capability can get PFNs.
   In 4.0 and 4.1 opens by unprivileged fail with -EPERM.  Starting from
   4.2 the PFN field is zeroed if the user does not have CAP_SYS_ADMIN.

Using phase 4 of Shibata's team, implement the program "large" that uses 2 times the physical memory (adjust memory size in qemu). Make sure the swap area is very large, greater than 4GB. This program declares a large vector. For each element of the vector, a new number is written sequentially. Make the program stop using getchar() after reading 1.5 times the physical memory so that many swaps must have happen.
Study /proc/$PID/pagemap to inform what are the pages in the swap partition as it is written:

If the page is not present but in swap, then the PFN contains an
   encoding of the swap file number and the page's offset into the
   swap. Unmapped pages return a null PFN. This allows determining
   precisely which pages are mapped (or in swap) and comparing mapped
   pages between processes.

Team 8: page table: mm_struct and vm_area_struct

Read https://manybutfinite.com/post/how-the-kernel-manages-your-memory/ . Google for any other related material.
mm_struct and vm_area_struct corresponds to a process page table.

Using phase 4 of Shibata's team, implement the program "large" that uses 1/3 of physical memory. The program "large" may declare a large vector with 1/3 of physical memory. For each vector position, write a number. The memory pages necessary to this large vector will be taken as the vector is being accessed. According to https://manybutfinite.com/post/how-the-kernel-manages-your-memory/, there is a vm_area_struct that contains the data of the process.
Execute "large" three times in background ($ large& $ large& $ large&). It is expected that a "page fault" happens.
Put a breakpoint when an old memory page is removed. Explain how this page was chosen.

Team 9: invasion

Using phase 4 of Shibata's team, implement the program "invade". Basically, it uses a pointer that reads a position in the memory. You may  point to somewhere valid in your code and then continually increment the pointer to a maximum; and again decrementing the pointer trying to reach zero. As the Linux kernel might be in the upper memory, an error must happen when the pointer is being incremented. Do you get any error? Insert a breakpoint closer to the interrupt vector when this error happens. Explain. Obs: observe the effect of different sizes of the physical memory in qemu. Explain how virtual memory was allocated by /proc/$PID/maps (try to block the process by a getchar beforer the error).
Tip: See data abort

Team 10: allocating a physical page frame

Using phase 4 of Shibata's team, implement the program "large" that uses 2 times the physical memory (adjust memory size in qemu). This program declares a large vector. For each element of the vector, a new number is written sequentially. 

Before running the code, insert a breakpoint to study when "large" will get a new physical page frame. Take care with false page allocations not related to "large".

Study: https://manybutfinite.com/post/how-the-kernel-manages-your-memory/ and the book: "Professional Linux Kernel Architecture" (that is outdated). Google for more information on how Linux chooses a new physical page frame when needed. 

Explain how  the new physical page frame is allocated and inserted in the process page table.