Assignment 4

Team 1: page fault on malloc

Using phase 4 of Shibata's team, write a program that continually gets memory (malloc) and writes on a structure, creating a linked list (where each cell is in this structure), i.e., a cell (this structure) must point to the former one.  After some time, due to virtual memory, data must go to disk in order to free memory to more data through do_swap_page. How does the Linux choose the page to evict? Let the program run freely. What error do you get? Insert a breakpoint closer to the interrupt vector when this error happens. Explain. Obs: observe the effect of differente sizes of the physical memory in qemu. Explain how virtual memory was allocated by /proc/$PID/maps (the process may be blocked by a getchar).

Team 2: allocating a physical page frame

Using phase 4 of Shibata's team, implement the program "large" that uses 2 times the physical memory (adjust memory size in qemu). This program declares a large vector. For each element of the vector, a new number is written sequentially. 

Before running the code, insert a breakpoint to study when "large" will get a new physical page frame. Take care with false page allocations not related to "large".

Study: https://manybutfinite.com/post/how-the-kernel-manages-your-memory/ and the book: "Professional Linux Kernel Architecture" (that is outdated). Google for more information on how Linux chooses a new physical page frame when needed. 

Explain how  the new physical page frame is allocated and inserted in the process page table.

Team 3: page table: physical address

In https://manybutfinite.com/post/how-the-kernel-manages-your-memory/ we read:

A program's VMAs are stored in its memory descriptor both as a linked list in the mmap field, ordered by starting virtual address, and as a red-black tree rooted at the mm_rb field. The red-black tree allows the kernel to search quickly for the memory area covering a given virtual address. When you read file /proc/pid_of_process/maps, the kernel is simply going through the linked list of VMAs for the process and printing each one.

Explain what it means. Locate in the linux kernel source where is the red-black tree is used to sort the virtual memory presentation. 

Using phase 4 of Shibata's team, implement the program "large" that uses 1/3 of physical memory. This program may declare a large vector that are always being writing in each of its elements. The memory pages necessary to this large vector will be taken as the vector is being accessed. Insert 1000 numbers in this vector and stop it by getchar(). 

Modify the presentation in proc/$PID/maps so that the physical page frame is also presented. Test it with your program: "large". 

Team 4: stack, recursion

Using phase 4 of Shibata's team, write the recursive function "recursive(x1,x2,x3)" which increments x1, x2, x3 and calls itself. The stack must grow until occurs a page_fault. Explain how does Linux treat this page_fault. Let the program run freely. What error do you get? Insert a breakpoint closer to the interrupt vector when this error happens. Explain. Obs: observe the effect of differente sizes of the physical memory in qemu. Explain how virtual memory was allocated by /proc/$PID/maps (the process may be blocked by a getchar).

Team 5: fork bomb

A fork bomb is a program that continually forks. Ex: while(1) {fork();};
Using phase 4 of Shibata's team, implement a fork bomb. You may get errors due to a run out of memory in the kernel or in the user space. Explain what error did you get. It is very likely that you got a run out of memory in the kernel. If so, modify the fork bomb in order to allocate memory before each fork (you may declare data as global data taking a lot of memory), so that, the user memory runs out before the kernel memory. In this case, you may get page faults that will swap data between memory and disk until a future fatal error happens. Study it.  Let the program run freely. What error do you get? Insert a breakpoint closer to the interrupt vector when this error happens. Explain. Obs: observe the effect of differente sizes of the physical memory in qemu. Explain how virtual memory was allocated by /proc/$PID/maps (the process may be blocked by a getchar).

Team 6: swapping

Using phase 4 of Shibata's team, implement the program "large" that uses 1/3 of physical memory. This program may declare a large global vector. For each vector position, a number is written. You may execute four instances of "large" in background (in a shell: # large&). Insert breakpoints to study how the swap is performed.  Let the program run freely. Do you get any error?  Insert a breakpoint closer to the interrupt vector when this error happens. Explain. Obs: observe the effect of differente sizes of the physical memory in qemu. Explain how virtual memory was allocated by /proc/$PID/maps (each process may be blocked by a getchar).

Team 7: /proc/$PID/pagemap

Read in:
https://www.kernel.org/doc/Documentation/vm/pagemap.txt

Since Linux 4.0 only users with the CAP_SYS_ADMIN capability can get PFNs.
   In 4.0 and 4.1 opens by unprivileged fail with -EPERM.  Starting from
   4.2 the PFN field is zeroed if the user does not have CAP_SYS_ADMIN.

Using phase 4 of Shibata's team, implement the program "large" that uses 2 times the physical memory (adjust memory size in qemu). This program declares a large vector. For each element of the vector, a new number is written sequentially. Make the program stop using getchar() after reading 1.5 times the physical memory so that many swaps must have happen.
Study /proc/$PID/pagemap to inform what are the pages in the swap partition as it is written:

If the page is not present but in swap, then the PFN contains an
   encoding of the swap file number and the page's offset into the
   swap. Unmapped pages return a null PFN. This allows determining
   precisely which pages are mapped (or in swap) and comparing mapped
   pages between processes.

Team 8: page table: mm_struct and vm_area_struct

Read https://manybutfinite.com/post/how-the-kernel-manages-your-memory/ . Google for any other related material.
mm_struct and vm_area_struct corresponds to a process page table.

Using phase 4 of Shibata's team, implement the program "large" that uses 1/3 of physical memory. This program may declare a large vector with 1/3 of physical memory. For each vector position, write a number. The memory pages necessary to this large vector will be taken as the vector is being accessed. According to https://manybutfinite.com/post/how-the-kernel-manages-your-memory/, there is a vm_area_struct that contains the data of the process. Put a breakpoint when an old memory page is removed. Explain how this page was chosen.

Team 9: invasion

Using phase 4 of Shibata's team, implement the program "invade". Basically, it uses a pointer that reads a position in the memory. You may  point to somewhere valid in your code and then continually increment the pointer to a maximum; and again decrementing the pointer trying to reach zero. As the Linux kernel might be in the upper memory, an error must happen when the pointer is being incremented. Do you get any error? Insert a breakpoint closer to the interrupt vector when this error happens. Explain. Obs: observe the effect of different sizes of the physical memory in qemu. Explain how virtual memory was allocated by /proc/$PID/maps (try to block the process by a getchar beforer the error).
Tip: See data abort

Team 10: shared memory

Using phase 4 of Shibata's team, implement a program that creates a shared memory greater than the physical memory. One process must continually and sequentially write on this shared memory and another, must read.  A page swap must happen after some writes due to the size of the shared memory. Insert a breakpoint to study the page fault and the swap. Explain. Obs: observe the effect of different sizes of the physical memory in qemu. Obs: observe the effect of different size of the physical memory in qemu. Explain how virtual memory was allocated by /proc/$PID/maps  (the process may be blocked by a getchar).