Access the redirection table entry in IO APIC

http://wiki.osdev.org/IOAPIC
http://wiki.osdev.org/APIC

These two URLs provides detailed the explanation about IOAPIC, but the read/write code do not work on my CentOS machine. The IO APIC base is 0xfec00000 on my machine (index 0xfec00000; port 0xfec00010). It is a memory maped IO memory.

I use the code in both user-level program and a kernel module. The code raises an segmentation fault. After looking at the dmesg, I notice that it is because page table does not know how to translate the virtual address (i.e., 0xfec00000).

Then I use the ioremap() in the kernel module, it works.

#include <linux/module.h>
#include <linux/init.h>

#include <asm/io.h>


MODULE_LICENSE("Dual BSD/GPL");



static int hello_init(void)
{
    // IO APIC base is 0xfec00000
    // IO APIC index: 0xfec00000
    // IO APIC data: 0xfec00010
    // The redicection table of the keyboard is at index 0x12
    // please see southbridge manual for redirection table
    // the original value of the redirection table entry is 0x939
    // We change it to 0xa39, which enables SMI triggering
    //
    // if you want to compile this kernel moudule,
    // please also change the Makefile to hello6.o
    //
    // just run the script enable_kb_SMI.sh to enable the SMI
    // triggering
    //
    unsigned int IRQ1_entry;
    void * IOAPIC_BASE = ioremap(0xfec00000, 32);
    iowrite8(0x12, IOAPIC_BASE);
    IRQ1_entry = ioread32((unsigned char*)IOAPIC_BASE + 0x10);
    printk("IRQ1 Entry Value is: %x\n", IRQ1_entry);
     
    iowrite8(0x12, IOAPIC_BASE);
    iowrite32(0xA39, (unsigned char*)IOAPIC_BASE + 0x10);
  
    printk("Successfully enable keyboard SMI triggering\n");
    return 0;
}

static void hello_exit(void)
{
    printk("Goodbye, crule world\n");
}

module_init(hello_init);
module_exit(hello_exit);

Flush the Instruction Cache

// flush the cache, eip is the current eip address

__asm__ __volatile__("wbinvd"); // write back cache and invalidate cache

__asm__ __volatile__(
"CLFLUSH (%0)"
:"=r"(eip));
wbinvd:

Writes back all modified cache lines in the processor’s internal cache to main memory and invalidates (flushes) the internal caches. The instruction then issues a special- function bus cycle that directs external caches to also write back modified data and another bus cycle to indicate that the external caches should be invalidated. 


clfush:

Invalidates the cache line that contains the linear address specified with the source operand from all levels of the processor cache hierarchy (data and instruction). The invalidation is broadcast throughout the cache coherence domain. If, at any level of the cache hierarchy, the line is inconsistent with memory (dirty) it is written to memory before invalidation


Please see intel manual instruction volume for more details

memory management in kernel

http://duartes.org/gustavo/blog/post/how-the-kernel-manages-your-memory

How to Limit the Amount of Memory Available to OS

Linux:

set the mem=2000m as the boot arguments, then the Linux only use first 2000MB memory. If the total RAM is 2G, it reserves 48MB.
http://fengweizhang.blogspot.com/2012/06/boot-arguments-on-linux.html


Windows:

Add line
MaxPhyPage=0x7D000 
at [386enh] section of C:\WINDOWS\system.ini
0x7D000 * 4KB = 2000MB
http://support.microsoft.com/kb/181862

Memory tool on Windows

vmmap
http://technet.microsoft.com/en-us/sysinternals/dd535533
cheatengine
http://www.cheatengine.org/

VMware fusion snapshot

VMware Fusion snapshot saves the whole memory image file when you take a snapshot. You can find the memory image in the VM folder.

Here are some more detailed info about VMware snapshot
http://www.vmware.com/support/ws4/doc/preserve_snapshot_ws.html

Disable PAE on Windows XP SP3

Go to computer -> properties, it will show in the hardware specification if the PAE enabled.
Enabling PAE (by setting bit 5, PAE, of the system register CR4), which is the sixth bit from the left. 

Disable method:

Normal boot.ini with PAE & DEP

multi(0)disk(0)rdisk(0)partition(1)\WINDOWS="Micro soft Windows XP Pro x86" /noexecute=optin /fastdetect

Disabled DEP & PAE

multi(0)disk(0)rdisk(0)partition(1)\WINDOWS="Micro soft Windows XP Pro x86" /execute /fastdetect

__________________

Process Address Space

1. Each process has a process descriptor, and the location is in Kernel Space (3G-4G)
e.g. virtual address: 0xF44F7930, physical address: 0x344F7930

2. Each process has a memory descriptor, and the location is in Kernel Space (3G-4G)
e.g. virtual address: 0xF6CEA900, physical address: 0x36CEA900

3. Each process has a CR3 value and page directory and page table are in Kernel Space (3G-4G). And the value in CR3 register is a physical address. and value in pgd field of mm_struct is a virtual address. And pgd = CR3 + PAGE_OFFSET
e.g. CR3: 0x36D1C000, pgd:0xF6D1C000

4. Static code of process is in User Space (0G- 3G). The physical address of static is non-continuous, and it is allocated by page (4K).
e.g. virtual address: 0x8048000 - 0x8048DBC

5. Different process could have same linear address space.
e.g. static code of Firefox: 0x8048000 - 0x806D333; static code of Minicom: 0x8048000 - 0x805E333

PAE

http://en.wikipedia.org/wiki/Physical_Address_Extension

ULK page 51

Experiments with the Linux Kernel: Process Segments

http://linuxgazette.net/112/krishnakumar.html

Understanding the kernel code, and CR3 register

Recently, I was working on the kernel code and CR3 register. This article is going to record what I have learn.

First Question: How can we get the kernel code of Operating System?

Get Virtual Address of kernel code:

$ cat /boot/System.map-2.6.32-33-generic | grep text

c0100000 T_text

The first one is start virtual address of kernel code

Get Physical Address of kernel code:

$ cat /proc/iomem

00100000-00590de6 : Kernel Code

More info: please see

http://fengweizhang.blogspot.com/2010/12/boot-cat-system.html

http://jianggmulab.blogspot.com/2009/09/blog-post.html

Second Question: How to translate virtual address to physical address? 

Control Register 3 (CR3) stores the base address of Page Directory Table. More info please see page 47 in ULK book. 

Third Question: Which CR3 I should use to translate VA to PA of kernel code?

Any CR3. For each process, it has its own address spaces and CR3. No matter which processes the operating system is running, the virtual address and physical address of kernel code will never change. Although the CR3 is different depends on the running process, same VA will map to same PA by CR3.

Each process has its own address space. From the virtual address point of view, it has some address space at 0-3G region, and it also has some address space at 3-4G region (this is for kernel mode). All Processes get same mappings for the kernel address space (3-4G). Each process also has its own page directory table and page tables. CR3 is the base address of page directory table. The address space at 3-4G region must have its own translation entries in page directory table and page tables. These entries are always same in different page directory table and page tables. 

Process 1 has virtual address space from 0xc0100000-0xc0590de6 (kernel code), translate to physical address is 0x00100000-0x00590de6

Process 2 has virtual address space from 0xc0100000-0xc0590de6 (kernel code), translate to physical address is 0x00100000-0x00590de6

Thus, process 1 and process 2 has same kernel memory mappings. (CR3 is different)

cat /proc/iomem

This is command shows the io memory map
This is the virtual address (linear address in Linux).
For example, it tells you the virtual address of kernel code.
If you want to get the kernel code, you could write a kenel module, and read the code out.


[root@localhost fengweiWorkSpace]# cat /proc/iomem
00010000-0009e3ff : System RAM
000a0000-000bffff : Video RAM area
000c0000-000cefff : Video ROM
000d3800-000d3fff : Adapter ROM
000f0000-000fffff : System ROM
00100000-7ddff7ff : System RAM
  00400000-00620dc5 : Kernel code
  00620dc6-007039fb : Kernel data
7ddff800-7de53bff : ACPI Non-volatile Storage
7de53c00-7de55bff : ACPI Tables
7de55c00-7dffffff : reserved
80000000-800fffff : PCI Bus #01
  80000000-8001ffff : 0000:01:00.0
  80020000-8002ffff : 0000:01:00.0
80100000-801fffff : PCI Bus #03
  80100000-8010ffff : 0000:03:00.0
  80110000-80110fff : 0000:03:02.0
80200000-8021ffff : 0000:00:19.0
  80200000-8021ffff : e1000e
80220000-80223fff : 0000:00:1b.0
  80220000-80223fff : ICH HD audio
80224000-80224fff : 0000:00:03.3
80225000-80225fff : 0000:00:19.0
  80225000-80225fff : e1000e
80226000-802263ff : 0000:00:1a.7
  80226000-802263ff : ehci_hcd
80226400-802264ff : 0000:00:1f.3
d0000000-dfffffff : PCI Bus #01
  d0000000-dfffffff : 0000:01:00.0
e0000000-fed003ff : reserved
fed20000-fed9ffff : reserved
fedad000-fedad00f : 0000:00:03.0
fee00000-feefffff : reserved
ff970000-ff9707ff : 0000:00:1f.2
  ff970000-ff9707ff : ahci
ff980800-ff980bff : 0000:00:1d.7
  ff980800-ff980bff : ehci_hcd
ffb00000-ffffffff : reserved

NAND and NOR Flash

NOR flash used as main memory
NAND flash used as secondary memroy.

In embeded device:
used as main memory: ROM, EPROM, NOR flash, SRAM, DRAM
used as secondary memory: NAND flash, SD, DOC(disk on chip).

NOR and NAND flash differ in two important ways:

• the connections of the individual memory cells are different
• the interface provided for reading and writing the memory is different (NOR allows random-access for reading, NAND allows only page access)

http://en.wikipedia.org/wiki/Flash_memory#Distinction_between_NOR_and_NAND_flash

NVRAM, DRAM, SRAM

Non-Volatile random access memory.
E.g. flash disk
it will not loose data after power off
Dynamic random access memory (DRAM) and Static random access memory (SRAM) will need power to keep data.
DRAM: volatile, need periodically refreshed (home PCs are DRAM)
SRAM: volatile

NVRAM fast than disk, but it more expensive

Access any memory by using kernel module

If you write a user program, it cannot read all the memory.

userprogram:

    virtualAddr = 0xc0010203 // randomly define virtual address

    for(i = 0; i< SIZE; i++)
    printf("%x", *(char*)(virtualAddr+i));

This program will give your segmentation fault when you run it.
Remember, CPU only understand virtual address.

Solution, write a kernel module to access any memory

#include <linux/module.h>
#include <linux/init.h>
#include <asm/io.h>

// 1G = 0x40000000
// trusted OS kernel code physical address is (0x400000, 0x620de5)
// add 1G, then test if it could access from this untrusted OS
#define PHYSTARTADDR 0x40400000
#define PHYENDADDR 0x40620de5
// this is the virtual address of kernel code
#define VIRSTARTADDR 0xc0400000
#define VIRENDADDR 0xc0620de5
#define SIZE 100

MODULE_LICENSE("Dual BSD/GPL");

static int hello_init(void)
{
    char mem[SIZE];
    int i;
    void *virtualAddr ;

//  if we want to access the memory range beyond current DIMM
//  we could use ioremap to create the pagetables, so cpu will understand

//  this virtual address.
//  virtualAddr = ioremap(PHYSTARTADDR, SIZE);

//  this case, we just want to read all the current memory range, 

//  which is DIMM has been initialized.
    virtualAddr = VIRSTARTADDR;

    for(i = 0; i< SIZE; i++)
    printk("%x", *(char*)(virtualAddr+i));
 
    printk("\nhello, world\n");
    return 0;
}
static void hello_exit(void)
{
    printk("Goodbye, crule world\n");
}
module_init(hello_init);
module_exit(hello_exit);

Memory System Configuration

OS: Cent OS 5.5
Chipset: AMDK8

you could get memory system configuration from following file:
/sys/bus/pci/devices/0000\:00\:18.0/  /* this is bus 0, device 18, function 0 registers */

E.g.
/* This is use hexdump to see the memory configuration */ 
hexdump /sys/bus/pci/devices/0000\:00\:18.0/config -v > ~/f0_conf
hexdump /sys/bus/pci/devices/0000\:00\:18.1/config -v > ~/f1_conf
hexdump /sys/bus/pci/devices/0000\:00\:18.2/config -v > ~/f2_conf
hexdump /sys/bus/pci/devices/0000\:00\:18.3/config -v > ~/f3_conf

For the meaning of these 256 bytes, please read the BIOS Kernel Development Guide(BKDG). There is another article about this

/dev/mem Read Physical Memory

This is the physical memory address, if you want to read/write physical memory, you can open this file using fopen() for open(), and read/write it. For example, you can read/write SMRAM at 0xa0000

However, it cannot read /dev/mem mmap after 0x101000, but you could use mmap to solve this problem. please see article. 

http://my.opera.com/yangguangxiang/blog/2008/02/17/dev-mem

在2.4，可以直接打开/dev/mem，然后读取。

在2.6，直接打开/dev/mem后，只可以读取前0x101000部分的内容(ubuntu)。大约是1MB加4KB。读取后面的内容将出现"open not permitted"错误。

解决的方法是使用mmap()。routine如下：

f = open("/dev/mem", O_RDONLY);

my_mem = mmap (0, 0x1000, PROT_READ, MAP_SHARED, f, 0x34f000);
if (my_mem == MAP_FAILED)
printf("map failed %s\n",strerror(errno));
通过my_mem就可以得到0x101000之后的内存内容了。

understanding mmap()

#include <sys/mman.h>

void *mmap(void *addr, size_t length, int prot, int flags, int fd, off_t offset);

The basic idea is to map the content of a file to memory.

void *addr: preferable addr as the start address in memory, usually set as 0, becuase the actually start address is returned by mmap().

size_t length: the number of bytes you want to map

off_t offset:  the start address of file you want to map. or you could say the offset of the current file you are going to map it to memory.

int fd:  the file descriptor you the file.

int prot: memory protection of the memory just mapped.
       The prot argument describes the desired memory protection of  the  map‐
       ping  (and  must  not  conflict with the open mode of the file).  It is
       either PROT_NONE or the bitwise OR of one  or  more  of  the  following
       flags:
       PROT_EXEC  Pages may be executed.
       PROT_READ  Pages may be read.
       PROT_WRITE Pages may be written.
       PROT_NONE  Pages may not be accessed.

int flags: mapped memory is shared with other process or private.

here is the example of mmap usage, but this is not working:

#include <stdio.h>
#include <stdlib.h>
#include <unistd.h>
#include <errno.h>
#include <sys/stat.h>
#include <fcntl.h>
#include <sys/mman.h>

#define STARTADDR 0x400000
#define ENDADDR 0x620de6
#define SIZE ENDADDR - STARTADDR

int main(int argc, char *argv[])
{
    int fd1 = open("/dev/mem", O_RDONLY);
    if(fd1 == -1)
    {
    fprintf(stderr, "open error\n");
    exit(1);
    } 
    int fd2 = open("staticMem", O_CREAT | O_RDWR);
    if(fd1 == -1)
    {
    fprintf(stderr, "open error\n");
    exit(1);
    }
    void* my_mem = mmap(0, SIZE, PROT_READ, MAP_SHARED, fd1, STARTADDR);
    if(my_mem == MAP_FAILED)
    {
    fprintf(stderr, "map failed\n");
    exit(1);
    } 
    if(write(fd2, my_mem, SIZE)!= SIZE)
    {
    fprintf(stderr, "write error\n");
    exit(1);
    }
    return 1;
}

logical address, linear address(virtual address), physical address

logical address: before segementation unit translate, in projected mode, it is same value as linear or virtual address. because linux does not use segmentation.

linear address or virtual address: Before paging unit

physical address: not necessary map to RAM, to could be mapped to other device(ROM, PCI device)

how to get the physicall address of kernel code

$ cat /boot/System.map-2.6.32-27-generic | grep text

This command will give you virtual address. The first one is the start of kernel code, For example, on my Ubuntu:

$ cat /boot/System.map-2.6.32-27-generic | grep text 

c0100000 T _text

If you change the c to 0, you changed the virtual address to physical address on Linux. 

Why? please see: http://jianggmulab.blogspot.com/2009/09/blog-post.html

$ cat /boot/System.map-2.6.32-27-generic | grep etext

This one will give you the end virtual address of kernel code.

$ cat /proc/iomem

This command will give your general picture of current memory, and all the address are physical address. For example, on my Ubuntu:
  00100000-00592a36 : Kernel code