Programming Assignment – 2CSE 421/521 – Operating SystemsDue: April 5th and April 19th @11:59 pm, 20191. PreparationBefore beginning your work, please read the following carefully:● Chapters 8-9 from Silberschatz (9th edition)● Lecture slides on Memory and Virtual Memory● Pintos Introduction● Pintos Reference Guide● Complete Pintos Documentation (PDF file) -- for reference only2. Task: Implement the User Programs of Pintos OSIn this project, you are asked to perform “kernel” level programming of the “UserPrograms” component in the Pintos operating system. The base code already supports loadingand running user programs, but no I/O or interactivity is possible. In this project, you will enableprograms to interact with the OS via system calls.Before beginning this assignment, make sure you read these sections from PintosReference Guide: section A.1 Pintos Loading, A.2 Threads, A.3 Synchronization, A.4 InterruptHandling, A.5 Memory Allocation, A.6 Virtual Addresses, and Appendix E (Debugging Tools).Page: 1/193. Setting Up The Pintos EnvironmentProject-2 does not depend on project-1. No code from project-1 is required for thisassignment. So, we suggest that you fetch a clean copy of the Pintos source code.We have prepared a Virtual Machine image of Pintos which will make theinstallation/configuration process much simpler for you. This will enable you to develop yourpintos project on any Linux, Windows, or Mac system (i.e. your own machine) without muchhassle. You can also download the VM image from:https://buffalo.box.com/s/erdn4wnde8fmyzk3y36lln4cxot4y649http://ftp.cse.buffalo.edu/CSE421/UB-pintos.ovaAfter downloading the file, verify the integrity of the file by comparing the md5 hash of thedownloaded file with:ecf501c3494f741f6d56f7ebe6064bebYou will need VirtualBox software for your host OS to run this VM. You can downloadVirtualBox at the following link:https://www.virtualbox.org/wiki/DownloadsFor the detailed instruction on how to setup this VM on your host OS, please see theEnvironment Setup on Piazza:http://www.piazza.com/class_profile/get_resource/jqh89ng55by1rd/jrr0l4wxmog6h5In VirtualBox software, from the File menu, choose Import Appliance and point to thedownloaded UB-pintos.ova. You do not need to change any other settings for this procedure.Choose UB-Pintos from the left pane, and click on Start, your virtual machine should nowboot. Here is the login information if needed:Username: os-classPassword: os-classTo learn how to run, debug and test Pintos code, please read the Pintos Introduction.Page: 2/194. Implementation of the ProjectYou will be working primarily in the “userprog” directory of the source tree for thisassignment. Compilation should be done in the “userprog” directory.4.1 BackgroundUp to now, all of the code you have run under Pintos has been part of the operatingsystem kernel. This means, for example, that all the test code from the last assignment ran as partof the kernel, with full access to privileged parts of the system. Once we start running userprograms on top of the operating system, this is no longer true. This project deals with theconsequences.We allow more than one process to run at a time. Each process has one thread(multi-threaded processes are not supported). User programs are written under the illusion thatthey have the entire machine. This means that when you load and run multiple processes at atime, you must manage memory, scheduling, and other state correctly to maintain this illusion.In the previous project, we compiled our test code directly into your kernel, so we had torequire specific function interfaces within the kernel. From now on, we will test your operatingsystem by running user programs. This gives you much greater freedom. You must make sure thatthe user program interface meets the specifications described here, but given that constraint youare free to restructure or rewrite kernel code however you wish.4.2 Source FilesThe easiest way to get an overview of the programming you will be doing is to simply goover each part youll be working with. In `userprog, youll find a small number of files, but here iswhere the bulk of your work will be:`process.c`process.hLoads ELF binaries and starts processes.`pagedir.c`pagedir.hA simple manager for 80x86 hardware page tables. Although you probably wont want tomodify this code for this project, you may want to call some of its functions. See Section 4.1.2.3[Page Tables] from Pintos Manual, for more information.`syscall.c`syscall.hWhenever a user process wants to access some kernel functionality, it invokes a systemcall. This is a skeleton system call handler. Currently, it just prints a message and terminates theuser process. In part 2 of this project you will add code to do everything else needed by systemcalls.`exception.cPage: 3/19`exception.hWhen a user process performs a privileged or prohibited operation, it traps into the kernelas an “exception or “fault. These files handle exceptions. Currently all exceptions simply print1a message and terminate the process. Some, but not all, solutions to project 2 require modifyingpage_fault() in this file.`gdt.c`gdt.hThe 80x86 is a segmented architecture. The Global Descriptor Table (GDT) is a table thatdescribes the segments in use. These files set up the GDT. You should not need to modify thesefiles for any of the projects. You can read the code if youre interested in how the GDT works.`tss.c`tss.hThe Task-State Segment (TSS) is used for 80x86 architectural task switching. Pintos usesthe TSS only for switching stacks when a user process enters an interrupt handler, as does Linux.You should not need to modify these files for any of the projects. You can read the code if youreinterested in how the TSS works.4.3 Using the File SystemYou will need to interface to the file system code for this project, because user programsare loaded from the file system and many of the system calls you must implement deal with thefile system. However, the focus of this project is not the file system, so we have provided asimple but complete file system in the `filesys directory. You will want to look over the`filesys.h and `file.h interfaces to understand how to use the file system, and especially its manylimitations.There is no need to modify the file system code for this project, and so we recommendthat you do not. Working on the file system is likely to distract you from this projects focus.You will have to tolerate the following limitations of the file system:● No internal synchronization. Concurrent accesses will interfere with one another. Youshould use synchronization to ensure that only one process at a time is executing filesystem code.● File size is fixed at creation time. The root directory is represented as a file, so thenumber of files that may be created is also limited.● File data is allocated as a single extent, that is, data in a single file must occupy acontiguous range of sectors on disk. External fragmentation can therefore become aserious problem as a file system is used over time.● No subdirectories.● File names are limited to 14 characters.● A system crash mid-operation may corrupt the disk in a way that cannot be repaired1 We will treat these terms as synonyms. There is no standard distinction between them, although Intel processormanuals make a minor distinction between them on 80x86.Page: 4/19automatically. There is no file system repair tool anyway.One important feature is included:● Unix-like semantics for filesys_remove() are implemented. That is, if a file is open whenit is removed, its blocks are not deallocated and it may still be accessed by any threadsthat have it open, until the last one closes it. See Section 3.4.2 FAQ [Removing an OpenFile] from Pintos Manual for more information.You need to be able to create a simulated disk with a file system partition. Thepintos-mkdisk program provides this functionality. From the `userprog/build directory, executepintos-mkdisk filesys.dsk --filesys-size=2. This command creates a simulated disk named`filesys.dsk that contains a 2 MB Pintos file system partition. Then format the file systempartition by passing `-f -q on the kernels command line: pintos -f -q. The `-f option causes thefile system to be formatted, and `-q causes Pintos to exit as soon as the format is done.Youll need a way to copy files in and out of the simulated file system. The pintos `-p(“put) and `-g (“get) options do this. To copy `file into the Pintos file system, use the command`pintos -p file -- -q. (The `-- is needed because `-p is for the pintos script, not for the simulatedkernel.) To copy it to the Pintos file system under the name `newname, add `-a newname: `pintos-p file -a newname -- -q. The commands for copying files out of a VM are similar, but substitute`-g for `-p.Incidentally, these commands work by passing special commands extract and append onthe kernels command line and copying to and from a special simulated “scratch partition. Ifyoure very curious, you can look at the pintos script as well as `filesys/fsutil.c to learn theimplementation details.Heres a summary of how to create a disk with a file system partition, format the filesystem, copy the echo program into the new disk, and then run echo, passing argument x.(Argument passing wont work until you implemented it.) It assumes that youve already built theexamples in `examples and that the current directory is `userprog/build:pintos-mkdisk filesys.dsk --filesys-size=2pintos -f -qpintos -p ../../examples/echo -a echo -- -qpintos -q run echo xThe three final steps can actually be combined into a single command:pintos-mkdisk filesys.dsk --filesys-size=2pintos -p ../../examples/echo -a echo -- -f -q run echo xIf you dont want to keep the file system disk around for later use or inspection, you caneven combine all four steps into a single command. The --filesys-size=n option creates atemporary file system partition approximately n megabytes in size just for the duration of thepintos run. The Pintos automatic test suite makes extensive use of this syntax:pintos --filesys-size=2 -p ../../examples/echo -a echo -- -f -q run echo xYou can delete a file from the Pintos file system using the rm file kernel action, e.g.pintos -q rm file. Also, ls lists the files in the file system and cat file prints a files contents to thedisplay.Page: 5/194.4 How User Programs WorkPintos can run normal C programs, as long as they fit into memory and use only thesystem calls you implement. Notably, malloc() cannot be implemented because none of thesystem calls required for this project allow for memory allocation. Pintos also cant run programsthat use floating point operations, since the kernel doesnt save and restore the processorsfloating-point unit when switching threads.The `src/examples directory contains a few sample user programs. The `Makefile in thisdirectory compiles the provided examples, and you can edit it to compile your own programs aswell. Some of the example programs will only work once projects 3 or 4 have been implemented.Pintos can load ELF executables with the loader provided for you in `userprog/process.c.ELF is a file format used by Linux, Solaris, and many other operating systems for object files,shared libraries, and executables. You can actually use any compiler and linker that output 80x86ELF executables to produce programs for Pintos. (Weve provided compilers and linkers thatshould do just fine.)You should realize immediately that, until you copy a test program to the simulated filesystem, Pintos will be unable to do useful work. You wont be able to do interesting things untilyou copy a variety of programs to the file system. You might want to create a clean reference filesystem disk and copy that over whenever you trash your `filesys.dsk beyond a useful state, whichmay happen occasionally while debugging.4.5 Virtual Memory LayoutVirtual memory in Pintos is divided into two regions: user virtual memory and kernelvirtual memory. User virtual memory ranges from virtual address 0 up to PHYS_BASE, which isdefined in `threads/vaddr.h and defaults to 0xc0000000 (3 GB). Kernel virtual memory occupiesthe rest of the virtual address space, from PHYS_BASE up to 4 GB.User virtual memory is per-process. When the kernel switches from one process toanother, it also switches user virtual address spaces by changing the processors page directorybase register (see pagedir_activate() in `userprog/pagedir.c). struct thread contains a pointer to aprocesss page table.Kernel virtual memory is global. It is always mapped the same way, regardless of whatuser process or kernel thread is running. In Pintos, kernel virtual memory is mapped one-to-one tophysical memory, starting at PHYS_BASE. That is, virtual address PHYS_BASE accessesphysical address 0, virtual address PHYS_BASE + 0x1234 accesses physical address 0x1234, andso on up to the size of the machines physical memory.A user program can only access its own user virtual memory. An attempt to access kernelvirtual memory causes a page fault, handled by page_fault() in `userprog/exception.c, and theprocess will be terminated. Kernel threads can access both kernel virtual memory and, if a userprocess is running, the user virtual memory of the running process. However, even in the kernel,an attempt to access memory at an unmapped user virtual address will cause a page fault.Page: 6/194.6 Typical Memory LayoutConceptually, each process is free to lay out its own user virtual memory however itchooses. In practice, user virtual memory is laid out like this:In this project, the user stack is fixed in size. Traditionally, the size of the uninitializeddata segment can be adjusted with a system call, but you will not have to implement this.The code segment in Pintos starts at user virtual address 0x08048000, approximately 128MB from the bottom of the address space. This value is specified in [SysV-i386] and has no deepsignificance.The linker sets the layout of a user program in memory, as directed by a “linker scriptthat tells it the names and locations of the various program segments. You can learn more aboutlinker scripts by reading the Scripts chapter in the linker manual, accessible via `info ld.To view the layout of a particular executable, run objdump (80x86) or i386-elf-objdump(SPARC) with the `-p option.Page: 7/194.7 Accessing User MemoryAs part of a system call, the kernel must often access memory through pointers providedby a user program. The kernel must be very careful about doing so, because the user can pass anull pointer, a pointer to unmapped virtual memory, or a pointer to kernel virtual address space(above PHYS_BASE). All of these types of invalid pointers must be rejected without harm to thekernel or other running processes, by terminating the offending process and freeing its resources.There are at least two reasonable ways to do this correctly. The first method is to verifythe validity of a user-provided pointer, then dereference it. If you choose this route, youll want tolook at the functions in `userprog/pagedir.c and in `threads/vaddr.h. This is the simplest way tohandle user memory access.The second method is to check only that a user pointer points below PHYS_BASE, thendereference it. An invalid user pointer will cause a page fault that you can handle by modifyingthe code for page_fault() in `userprog/exception.c. This technique is normally faster because ittakes advantage of the processors MMU, so it tends to be used in real kernels (including Linux).In either case, you need to make sure not to “leak resources. For example, suppose thatyour system call has acquired a lock or allocated memory with malloc(). If you encounter aninvalid user pointer afterward, you must still be sure to release the lock or free the page ofmemory. If you choose to verify user pointers before dereferencing them, this should bestraightforward. Its more difficult to handle if an invalid pointer causes a page fault, becausetheres no way to return an error code from a memory access. Therefore, for those who want to trythe latter technique, well provide a little bit of helpful code:/* Reads a byte at user virtual address UADDR.UADDR must be below PHYS_BASE.Returns the byte value if successful, -1 if a segfaultoccurred. */static intget_user (const uint8_t *uaddr){int resu
