Sunday, January 22, 2006

Interrupt Handling Internals in Linux Kernel

*************************************************
Interrupt Handling Internals in Linux Kernel
*************************************************

===========================
Author: Gaurav Dhiman
Email : gauravd.chd@gmail.com
gaurav4lkg@gmail.com
gauravd_chd@yahoo.com
===========================


Index:
=====
- Introduction
- CPU Support for Handling Interrupts
- Details of Programmable Interrupt Controller
- Hardware checks performed by CPU
- Details of Interrupt Descriptor Table
- Task Gates
- Trap Gates
- Interrupt Gates
- Hardware checks for Interrupts
- Kernel Support for Handling Interrupts
- Low Level Interrupt Stubs
- Details of do_IRQ() function, core of Inteuupt Handling


Introduction
==========

This article talks about internal details of Interrupt Handling in Linux Kernel. This will discuss, the hardware prospective of interrupt handling from CPU, Linux Kernel's Interrupt Routing subsystem, Device Drivers's role in Interrupt handling.

Term Interrupt is self defined, Interrupts are signals sent to CPU on an INTR bus (connected to CPU) whenever any device want to get attention of CPU. As soon as the interrupt signal occurs, CPU defer the current activity and service the interruptby executing the interrupt handler corresponding to that interrupt number (also know as IRQ number).

One of the clasifications of Interrupts can be done as follows:
- Synchronous Interrupts (also know on as software interrupts)
- Asynchronous Interrupts (also know as hardware interrupts)

Basic difference between these is that, synchronous interrupts are generated by CPU's control unit on facing some abnormal condition; these are also know as exception in Intel's termenology. These are interrupts whihc are generated by CPU itself either when CPU detects an abnormal condition or CPU executes some of the special instructions like 'int' or 'int3' etc. on other hand, asynchronous interupts are those, which actually are generated by outside world (devices connected to CPU), As these interrupts can occur at any point of time, these are known as asynchronous interrupts.

Its important to note that both synchornous and asynchronous interrupts are handled by CPU on the completion of insturctionduring which the interrupt occur. Execution of a machine instruction is not done in one single CPU cycle, it take some cycles to complete. Any interrupt occurs in between the execution of instruction, will not be handled imediately, rather CPU will check of interrupts on the completion of instruction.


CPU support for handling interrupts
=============================

For handling interrupts there are few of the things which we expect the CPU to do on occurence of every interrupt. Wheneveran interrupt occurs, CPU performs some of the hardware checks, which are very much needed to make the system secure. Beforeexplaining the hardware checks, we will understand how the interrupts are routed to the CPU from hardware devices.


Details of Programmable Interrupt Controller
----------------------------------------------------------------

On Intel architecture, system devices (device controllers) are connected to a special device known as PIC (Programmable Interrupt Controller). CPU have two lines for receiving interrupt signals (NMI and INTR). NMI line is to recieve non-maskable interrupts; the interrupts which can not be masked, means which can not be blocked at any cost. These interrupts are of hightest priority and are rarely used. INTR line is the line on which all the interrupts from system devices are received. These interrupts can be masked or blocked. As all the interrupt signals need to be multiplxed on single CPU line, we need some mechanisum through which interrupts from different device controllers can be routed to single line of CPU. This routing or multiplexing is done PIC (Programmable Interrupt Controller). PIC sits between system devices and CPU and have multiple input lines; each line connected to different divice contollers in system. On other hand IPC have only one output line which is connected to the CPU's INTR line on which it sends signal to CPU.

There are two PIC controllers joined together and the output of second PIC controller is connected to the second input of first PCI. This setup allows maximum of 15 input lines on which different system device controllers can be connected. PIC have some programmable registers, through which CPU communicates with it (give command, mask/unmask interrup lines, read status). Both PICs have their own following registers:

- Mask Register
- Status Register

Mask register is used to mask/unmask a specific interrupt line. CPU can ask the PIC to mask (block) the specific interrupt by setting the corresponding bit in mask register. Unmasking can be done by clearing that bit. When a particular interrupt is being masked, PIC do receive the interrupts on its corresponding line, but do not send the interrupt to CPU in whihc case tCPU keps on doing what it was doing. When an interrupts are being masked, they are not lost, rather PIC remembers those anddo send the interrupt to CPU when CPU unmasks that interrupt line. Masking is different from blocking all the interrupts toCPU. CPU can ignore all the interrupts coming on INTR line by clearing the IF flag in EFLAGS register of CPU. When this bitis cleared, interrupts coming on INTR line are simply ignored by CPU, we can consider it to be blocking of interrupts. So now we understand that masking is done at PIC level and individual interrupt lines can be masked or unmasked, where as blocking is done at CPU level and is done for all the interrupts except NMI (Non-Maskable Interrupt), which is received on NMI line of CPU and can not be blocked or ignored.

Now days, interrupt architecture is not as simple as shown above. Now days machines uses the APIC (Advanced Programmable Interrupt Controller), which can support upto 256 interrupt lines. Along with APIC, every CPU also have inbuilt IO-APIC. We wont go into details of these right now (will be covered in future articles).


Hardware checks performed by CPU
----------------------------------------------------

Once the interrupt signal is received by CPU, CPU performs some hardware checks for which no software machine instructions are executed. Before looking into what these checks are, we need to understand some architecture spcific data structures maintained by kernel.

Details of Interrupt Descriptor Table
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

Kernel need to maintain one IDT (Interrupt Descriptor Table), which actually maps the interrupt line with the interrupt handler routine. This table is of 256 enteries and each entry is of 8 bytes. First 32 enteries of this table are used for exceptions and rest are used for hardware interrupts received from outer world. This table can contain three different type of enteries; these three different types are as follows:

- Task Gates
- Trap Gates
- Interrupt Gates

Lets see what these gates are where these are used.

Task Gates
##########

Format of task gate entry is as follows:
- 0-15 bits ---- reserved (not used)
- 16-31 bits ---- points to the TSS (Task State Segment) entry of the process to which we need to switch.
- 32-39 bits ---- these bits are reserved and are not currently used.
- 40-43 bits ---- specify the type of entry (its value for task gate is 0101)
- 44th bit ---- always 0, not used
- 45-46 bits ---- this specifies the DPL (Decsriptor Previlege Level) level of gate entry.
- 47th bit ---- specifies if this entry is valid or not (1 - valid, 0 - invalid)
- 48-63 bits ---- reserved (not used)

Basically the task gates are used in IDT, to allow the user processs to make a context switch with another process without requesting the kernel to do this. As soon as this gate is hit (interrupt received on line for which there is a task gate in IDT), CPU saves the context (state of processor registers) of currently running process to the TSS of current process, whoseaddress is saved in TR (Task Register) of CPU. After saving the context of current process, CPU sets the CPU registers withthe values stored in the TSS of new process, whose pointer is saved in the 16-31 bits of the task gate. Once the registers are set with these new values, processor gets the new process and the context switch is done. Linux do not use the task gates, it only uses the trap and interrupt gates in IDT. So I will not explain the task gates any more.

Trap Gates
##########

Format of trap gates is as follows:
- 0-15 bits ---- first 16 bits of a pointer to a kernel function which need to be invoked when this gate is hit
- 16-31 bits ---- indicates the index of segment descriptor in GDT (Global Descriptor Table)
- 32-36 bits ---- these bits are reserved and are not currently used.
- 37-39 bits ---- always 000, not used
- 40-43 bits ---- specify the type of entry (its value for trap gate is 1111)
- 44th bit ---- always 0, not used
- 45-46 bits ---- this specifies the DPL (Decsriptor Previlege Level) level of gate entry.
- 47th bit ---- specifies if this entry is valid or not (1 - valid, 0 - invalid)
- 48-63 bits ---- last 16 bits of a pointer to a kernel function which need to be invoked when this gate is hit

Trap gates are basically used to handle exceptions generated by CPU. 0-15 bits and 48-63 bits together form the pointer (offset in segment identified by 16-31 bits of this entry) to a kernel function. The only difference between trap gates and interrupt gates is that, whenever an interrupt gate is hit, CPU automatically disables the interrupts by clearing the IF flag in CPU's EFLAG register, whereas in case of trap gate this is not done and interrupts remain enabled. As mentioned earlier trap gates are used for exceptions, so first 32 enteries in IDT are initialized with trap gates. In addition to this Linux Kernel also uses the trap gate for system call entry (entry 128 of IDT).

Interrupt Gates
#############

Format of interrupt gates is same as trap gates explained above, expect the value of type field (40-43 bits). In case of trap gates this have a value 1111 and in case of interrupts its 1110.

Format is as follows:
- 0-15 bits ---- first 16 bits of a pointer to a kernel function which need to be invoked when this gate is hit
- 16-31 bits ---- indicates the index of segment descriptor in GDT (Global Descriptor Table)
- 32-36 bits ---- these bits are reserved and are not currently used.
- 37-39 bits ---- always 000, not used
- 40-43 bits ---- specify the type of entry (its value for interrupt gate is 1110)
- 44th bit ---- always 0, not used
- 45-46 bits ---- this specifies the DPL (Decsriptor Previlege Level) level of gate entry.
- 47th bit ---- specifies if this entry is valid or not (1 - valid, 0 - invalid)
- 48-63 bits ---- last 16 bits of a pointer to a kernel function which need to be invoked when this gate is hit

Note: whenever the interrupt gate is hit, interrupts are disabled automatically.


Hardware Checks for Interrupts and Exceptions
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

Whenever an exception or interrupt occurs, corresponding trap/interrupt gate is hit and CPU performs some checks with fields of these gates. Things done by CPU are as follows:

1). get the ith entry from IDT (physical address and size of IDT is stored in IDTR register of CPU), here 'i' means the interrupt number.

2). read the segment descriptor index from 16-31 bits of IDT entry, lets say this to be 'n'

3). gets the segment descriptor from 'n'th entry in GDT (physical address and size of GDT is stored in GDTR register of CPU)

4). DPL of the nth entry in the GDT should be less that equal to CPL (Current Previelge Level, specified in the read-only lowermost two bits of CS register). Incase DPL > CPL, CPU will generate general protection exception. We will see ahead, whatdoes this check mean and why this is done. Simply saying:
a). DPL (of GDT entry) <= CPL ----- ok, switches stack if DPL < CPL
b). DPL (of GDT entry) > CPL ----- general protection exception
If DPL (of GDT entry) < CPL, we are entering the higher previlege level (probably from user to kernel mode). In this case CPU switches the hardware stack (SS and ESP registers) from currently running process's user mode stack to its kernel mode stack. We will see ahead, how this stack switch is exactly done. Note: stack switching idea has been mentioned here, but it actually happens after the 5th step mentioned below.

5). for software interrupts (generated by assembly instructions 'int'), one more check is done. This check is not performedfor hardware interrupts (interrupts generated by system devices and forwarded by PIC). Simply saying:
a). DPL (of IDT entry) >= CPL ---- ok, we have permission to enter through this gate
b). DPL < CPL ---- genreal protection exception

6). switches the stack if DPL (of GDT entry) < CPL. In addition to this mode of CPU (least significant two bits of CS) are also changed from CPL to DPL (of GDT entry)

7). if the stack switch has taken place (SS and ESP registers reset to kernelstack), then pushes the old values of SS and ESP (pointing to user stack) on this new stack (kernel stack)

8). pushes the EFALGS, CS and EIP registers on the stack (note: now we are working on kernel stack). This actually saves the pointer to user application instruction to which we need to return back after servicing the interrupt or exception

9). In case of exceptions, if there is any harware code, processor pushes that also on kernel stack

10). loads the CS with the value of GDT entry and EIP with the offset entry of IDT (0-15 bits + 48-63 bits)

All the above action is done by CPU hardware without the execution of any software instruction. Checks performed at step 4th and 5th (mentioned above) are important.

4th checks make sure that the code we are going to execute (Interrupt Service Routine) does not fall in a segment with lesser previlege. Obivously the ISR can not be in lesser previlege segment that what we are into. DPL or CPL can have 4 values (0,1,2 for kernel mode and 3 fo user mode). Out of these four only two are used, that is 0 (for kernel mode) and 3 (for user mode).

5th check makes sure that application can enter the kernel mode through specific gaes only, in Linux only through 128th gate entry which is for system call invocation. If we set the DPL field of IDT entry to be 0,1 or 2, application programme (running with CPL 3) cannot enter through that gate entry. If it tries, CPU will generate general protection exception. This is the reason that in Linux, DPL fields of all the IDT enteries (except 128th entry used for system call) are initialized with value '0', this makes sure only kernel code can access these gates not application code. In Linux 128th entry (used for system call) is of trap gate type and its DPL value is initialized to 3, so that application code can enter through this gate byassembly instruction "int 0x80"

Now lets see how does the stack switch happens when the DPL (of GDT entry) < CPL. CPU have TR (Task Register) register, which actually points to the TSS (Task Sate Segment) od currently running process. TSS is an architecture defined data structure which contains the stae of processor registers whenever context switch of this process happens. TSS include three sets of ESS and ESP fields, one for each level of processor (0,1 and 2). These fields specifies the stack to be used whenevr we entert that processor level. Lets say the DPL value in GDT entry is 0, in this case, CPU will load the SS register with the value of SS field in TSS for 0 level and ESP register with the value of ESP field in TSS for 0 level. After loading the SS and ESP with these values, CPU starts pointing to the new kernel level stack o current process. Old values of SS and ESP (CPU remembers them somehow) are now pushed on this new kernel level stack; this is done as we need to return back to old stack oncewe service the interrupts, exception or system call. Prudent readers must be wondering, why there is no firld for level 3 stack in TSS. Well the reason for this is that we never use the CPU's stack switching mechanism to switch from higher CPU level (kernel mode - 0,1 and 2) to lower CPU level (user mode - 3). This is the reason that CPU while entering the higher level(kernel mode) saves the previously used lower level stack (user mode) on the kernel stack.

Once all this CPU action is done, CPU's CS and EIP registers are pointing to the kernel functions written for handling interrupts or exceptions. CPU simply start executing the instructions at this point (now we are in kernel mode - level 0)


Kernel Support for Handling Interrupts
================================

In this section, we will be covering and walk through the kernel code executed in interrupt context. I will be reffering the the code as per 2.4.18 release of kernel.


Low Level Interrupt Stubs
~~~~~~~~~~~~~~~~~~~~~

Whenever an interrupt occurs, CPU performs the above mentioned hardware checks and start executing the following assembly instructions in kernel, whose pointer (offest in kernel code segment) is stored correstonding IDT entry.

File: include/asm-i386/hw_irq.h
###########################################################

155 #define BUILD_COMMON_IRQ() 156 asmlinkage void call_do_IRQ(void); 157 __asm__( 158 "\n" __ALIGN_STR"\n" 159 "common_interrupt:\n\t" 160 SAVE_ALL 161 SYMBOL_NAME_STR(call_do_IRQ)":\n\t" 162 "call " SYMBOL_NAME_STR(do_IRQ) "\n\t" 163 "jmp ret_from_intr\n");


175 #define BUILD_IRQ(nr) 176 asmlinkage void IRQ_NAME(nr); 177 __asm__( 178 "\n"__ALIGN_STR"\n" 179 SYMBOL_NAME_STR(IRQ) #nr "_interrupt:\n\t" 180 "pushl $"#nr"-256\n\t" 181 "jmp common_interrupt");

###########################################################

This macros is used at the kernel initialization time to write out the lowest interrupt stubs, which can be called from IDTby saving there offsets (pointers) in IDT gates. Kernel maintains one global array of function pointers (name of array - interrupt) in which it stores the pointer of these stubs. Code related to creation of these stubs (using above mentioned BUILD_IRQ macro) and saving their pointers in the global array "interrupt[NR_IRQS]" can be seen in file "arch/x86_64/kernel/i8259.c". In this file you will see the usage of BUILD_IRQ macro to create the interrupt stubs as follows:

File: arch/i386/kernel/i8259.c
###########################################################

40 #define BI(x,y) 41 BUILD_IRQ(x##y)
42
43 #define BUILD_16_IRQS(x) 44 BI(x,0) BI(x,1) BI(x,2) BI(x,3) 45 BI(x,4) BI(x,5) BI(x,6) BI(x,7) 46 BI(x,8) BI(x,9) BI(x,a) BI(x,b) 47 BI(x,c) BI(x,d) BI(x,e) BI(x,f)
48
49 /*
50 * ISA PIC or low IO-APIC triggered (INTA-cycle or APIC) interrupts:
51 * (these are usually mapped to vectors 0x20-0x2f)
52 */
53 BUILD_16_IRQS(0x0)
54
55 #ifdef CONFIG_X86_IO_APIC
56 /*
57 * The IO-APIC gives us many more interrupt sources. Most of these
58 * are unused but an SMP system is supposed to have enough memory ...
59 * sometimes (mostly wrt. hw bugs) we get corrupted vectors all
60 * across the spectrum, so we really want to be prepared to get all
61 * of these. Plus, more powerful systems might have more than 64
62 * IO-APIC registers.
63 *
64 * (these are usually mapped into the 0x30-0xff vector range)
65 */
66 BUILD_16_IRQS(0x1) BUILD_16_IRQS(0x2) BUILD_16_IRQS(0x3)
67 BUILD_16_IRQS(0x4) BUILD_16_IRQS(0x5) BUILD_16_IRQS(0x6) BUILD_16_IRQS(0x7)
68 BUILD_16_IRQS(0x8) BUILD_16_IRQS(0x9) BUILD_16_IRQS(0xa) BUILD_16_IRQS(0xb)
69 BUILD_16_IRQS(0xc) BUILD_16_IRQS(0xd)
70 #endif
71
72 #undef BUILD_16_IRQS
73 #undef BI

###########################################################

Above code actually creates the interrupt stubs and do not place there pointers in interrupt[NR_IRQS] array. The code whichplaces the pointers of these stubs in global array is as follows and can be found in same file "arch/x86_64/kernel/i8259.c"


File: arch/i386/kernel/i8259.c
###########################################################

100 #define IRQ(x,y) 101 IRQ##x##y##_interrupt
102
103 #define IRQLIST_16(x) 104 IRQ(x,0), IRQ(x,1), IRQ(x,2), IRQ(x,3), 105 IRQ(x,4), IRQ(x,5), IRQ(x,6), IRQ(x,7), 106 IRQ(x,8), IRQ(x,9), IRQ(x,a), IRQ(x,b), 107 IRQ(x,c), IRQ(x,d), IRQ(x,e), IRQ(x,f)
108
109 void (*interrupt[NR_IRQS])(void) = {
110 IRQLIST_16(0x0),
111
112 #ifdef CONFIG_X86_IO_APIC
113 IRQLIST_16(0x1), IRQLIST_16(0x2),
IRQLIST_16(0x3),
114 IRQLIST_16(0x4), IRQLIST_16(0x5), IRQLIST_16(0x6),
IRQLIST_16(0x7),
115 IRQLIST_16(0x8), IRQLIST_16(0x9), IRQLIST_16(0xa),
IRQLIST_16(0xb),
116 IRQLIST_16(0xc), IRQLIST_16(0xd)
117 #endif
118 };
119
120 #undef IRQ
121 #undef IRQLIST_16

###########################################################

Above code actually filles the global array of function pointers (array name interrupt[NR_IRQS]). Once the global array is nitialized with the pointers to interrupt stubs, we initialize the IDT (Interrupt Descriptor Table) in function "init_IRQ()"using this global array as follows:


File: arch/i386/kernel/i8259.c, Function: init_IRQ()
###########################################################

for (i = 0; i < (NR_VECTORS - FIRST_EXTERNAL_VECTOR); i++) {
int vector = FIRST_EXTERNAL_VECTOR + i;
if (i >= NR_IRQS)
break;
if (vector != IA32_SYSCALL_VECTOR && vector != KDB_VECTOR) {
set_intr_gate(vector, interrupt[i]);
}
}

###########################################################

In above loop, we loop over all the IDT enteries staring from "FIRST_EXTERNAL_VECTOR" (32, because first 32 enteries are for exception) and call "set_intr_gate()" function which actually set the interrupt gate descriptor. For entry 128, which is for system call invocation, interrupt gte is not set, for this rather trap gate is set and that is done in function trap_init(). In the same function init_IRQ(), after this looping, we initialize the IPI (Interprocessor Interrupts). These interruptsare sent from one CPU to another CPU in SMP machines.

Now we can see once these IDT eneries are set, whenever an interrupt occurs, CPU directly jumps to the code given in BUILD_IRQ macro. Now lets analyse what this macro do. Following is the code for BUILD_IRQ macro:


File: include/asm-i386/hw_irq.h
###########################################################

#define BUILD_IRQ(nr) asmlinkage void IRQ_NAME(nr); __asm__( "\n.p2align\n" "IRQ" #nr "_interrupt:\n\t" "push $" #nr "-256 ; " "jmp common_interrupt");

###########################################################

This assembly code first subtracts the IRQ number from 256 and pushes the result on kernel stack. After doing this it jumpsto "common_interrupt" assembly label, which simply saves the context of interrupted process (CPU resigters) on to kernel stack and then calls the C language function "do_IRQ()".


Details of do_IRQ() function, core of Inteuupt Handling
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

do_IRQ() is the common function to all hardware interrupts. This function is the most important to understand from the prespective of interrupt handling. We will first show the code of whole function and then explain it line by line in coming paragraphs with line refferences.


File: arch/i386/kernel/irq.c
###########################################################

563 asmlinkage unsigned int do_IRQ(struct pt_regs regs)
564 {
565 /*
566 * We ack quickly, we don't want the irq controller
567 * thinking we're snobs just because some other CPU has
568 * disabled global interrupts (we have already done the
569 * INT_ACK cycles, it's too late to try to pretend to the
570 * controller that we aren't taking the interrupt).
571 *
572 * 0 return value means that this irq is already being
573 * handled by some other CPU. (or is disabled)
574 */
575 int irq = regs.orig_eax & 0xff; /* high bits used in ret_from_
code */
576 int cpu = smp_processor_id();
577 irq_desc_t *desc = irq_desc + irq;
578 struct irqaction * action;
579 unsigned int status;
580
581 kstat.irqs[cpu][irq]++;
582 spin_lock(&desc->lock);
583 desc->handler->ack(irq);
584 /*
585 REPLAY is when Linux resends an IRQ that was dropped earlier
586 WAITING is used by probe to mark irqs that are being tested
587 */
588 status = desc->status & ~(IRQ_REPLAY | IRQ_WAITING);
589 status |= IRQ_PENDING; /* we _want_ to handle it */
590
591 /*
592 * If the IRQ is disabled for whatever reason, we cannot
593 * use the action we have.
594 */
595 action = NULL;
596 if (!(status & (IRQ_DISABLED | IRQ_INPROGRESS))) {
597 action = desc->action;
598 status &= ~IRQ_PENDING; /* we commit to handling */
599 status |= IRQ_INPROGRESS; /* we are handling it */
600 }
601 desc->status = status;
602
603 /*
604 * If there is no IRQ handler or it was disabled, exit early.
605 Since we set PENDING, if another processor is handling
606 a different instance of this same irq, the other processor
607 will take care of it.
608 */
609 if (!action)
610 goto out;
611
612 /*
613 * Edge triggered interrupts need to remember
614 * pending events.
615 * This applies to any hw interrupts that allow a second
616 * instance of the same irq to arrive while we are in do_IRQ
617 * or in the handler. But the code here only handles the _second_
618 * instance of the irq, not the third or fourth. So it is mostly
619 * useful for irq hardware that does not mask cleanly in an
620 * SMP environment.
621 */
622 for (;;) {
623 spin_unlock(&desc->lock);
624 handle_IRQ_event(irq, ®s, action);
625 spin_lock(&desc->lock);
626
627 if (!(desc->status & IRQ_PENDING))
628 break;
629 desc->status &= ~IRQ_PENDING;
630 }
631 desc->status &= ~IRQ_INPROGRESS;
632 out:
633 /*
634 * The ->end() handler has to deal with interrupts which got
635 * disabled while the handler was running.
636 */
637 desc->handler->end(irq);
638 spin_unlock(&desc->lock);
639
640 if (softirq_pending(cpu))
641 do_softirq();
642 return 1;
643 }

###########################################################

Here is the detailed explaination of do_IRQ() function, this has been
explained below line by line.

Line - 575 to 577
Get the number of the interrupt that got triggered. Its pushed on the kernel stack before pushing the context of the interrupted process. Get the processor or CPU id o which this code is being executed or in other means the CPU id of processor handling this interrupt. Get the pointer to the IRQ descriptor. IRQ descriptor is a kernel data structure which actually binds together the different ISRs (Interrupt Service Routines) registere by device drivers for same IRQ line. As mentioned earlieralso, same IRQ line can b shared between different devices, so their device drivers need to register their own ISRs to handle the interrupts genetated by these devices. IRQ descriptor data structure is defined as follows:

typedef struct {
unsigned int status;
hw_irq_controller *handler;
struct irqaction *action;
unsigned int depth;
spinlock_t lock;
} ____cacheline_aligned irq_desc_t;

Following is the significance of different elements in this stucture:

- status : Its a bit mask of different flags to identify the state of a particular IRQ line. We will see the use of differnet flags ahead in this article.

- handler : This is the pointer to the structure, whose each element is the pointer to the function related to the handlingof physical PIC (programmable interrupt controller). These functions are used to mask/unmask particular interrup line in PIC or to acknowledge the interrupt to PIC. The definitions of these PIC related functions can be found in file "arch/i386/kernel/i8259.c"

- action : This element is the pointer to the list o ISRs registered by different device drivers for this IRQ line. When a device driver registers its ISR to kernel using kernel function "irq_request()", the ISR is added to this list for that particular IRQ line.

- lock : This is spinlock to handle the synchronization problem while accessing any element in IRQ descriptor. Kernel execution context access the different elements of IRQ descriptor, but before doing so they should acquire this spinlock so that the synchronization can be maintained.

Line - 581 to 583
Here we increment the interrupt count received by this CPU, this is maintained for accounting purpose. Hold the spinlock before accessing any element of the IRQ descriptor for our interrupt line. We also mask and acknowledge the interrupt to PIC using handler function of our IRQ descriptor.

Line - 588 to 589
Now we clear the IRQ_REPLAY and IRQ_WAITING flags from ou IRQ descriptor flag. As mentioned earlier this is used to maintain the status of an interrupt handling line. We clear these flags because now we are going to handle this interrupt will not be anymore in reply or waiting mode. Actually IRQ_WAITING flagis used by device drivers in conjunction with IRQ_AUTODETECT flag for auto-detecting the IRQ line to which their device is connected. Device drivers use the probe_irq_on() function, which actually sets he IRQ_AUTODETECT and IRQ_WAITING flag for all the IRQ descriptors for whome no ISR has yet been registered.After calling probe_irq_on() function, device driver instructs the device to trigger an interrupt and then calls probe_irq_off(0 function. probe_irq_off() function actually looks for those IRQ descriptors whose IRQ_AUTODETECT flag is still set butIRQ_WAITING flag has been cleared. and returns the IRQ line number to device driver.

After clearing the IRQ_REPLAY and IRQ_WAITING flags in do_IRQ() function we set the IRQ_PENDING function. This is done, to indicate that we are planning to handle this interrupt if this interrupt is not disabled or not bein already handled by another CPU (in case of SMP machines). The use of setting IRQ_PENDING flag is explained in details in next few lines.

As we have see the interrupt and want to handle it by calling the set of ISRs (Interrupt Serive Routines) registered by different device drivers. We set IRQ_PENDING flag because seeing an interrupt does not mean we will for sure handle it. IRQ_PENDING flag helps us in following two cases:

- In case interrupt is disabled (set flag IRQ_DISABLED), we will not service the interrupt and will just keep it marked as pending (set flag IRQ_PENDING). Once the interrupt is again enabled (clear flag IRQ_DISABLED), ISRs will be called to service the interrupt. So IRQ_PENDING helps us to remember the intterupt which occured while that interrupt was disabled due to some reason.

Note: Here disabling interrupt does not mean masking a particular line at PIC level or disabling all the interrupt at CPU level by clearing the IF flag of CPU EFLAG register. Disabling here means the kernel has been asked not to service the interrupt, but the hardware triggering of interrupt signal is not being stopped at all.

- In case another CPU is already handling the previous interrupt requests on this IRQ line. In this case flag IRQ_INPROGRESS will already be set by that another CPU. Our role will be to just mark the interrupt as IRQ_PENDING and in away asks that other CPU to service this interrupt request also. When that CPU will finish its handling of previous interrupt, it will check this flag. Because of this flag being set by us, that CPU will again go and call all the ISRs once agian to service interrupt request we received on this IRQ line.

Line - 595 to 601
Now we check if this interrupt is not disabled (flag IRQ_DISABLED is clear) and at the same time is also not being handled by another CPU (flag IRQ_INPROGRESS is also clear), we go forward and clear the IRQ_PENDING flag and sets the IRQ_INPROGRESSflag to indicate that we take the responsibility of handling this interrupt request. Now while we are handling this interrupt request, lets sa another CPU receives an interrupt on same IRQ line, that CPU will simple mark the IRQ_PENDING flag and will transfer his responsibility to us and in that case we (CPU we are executing on) will be responsible to serve that interrupt request also.

Line - 609 to 610
If there is no registered ISR for this IRQ line, we simply return from interrupt context after releasig the lock we hold and serving the softirqs (if any pending).

Line - 622 to 630
Now we are al set to call the registered ISRs (device driver's functions), so that they can figure out which device connected to this IRQ line has actually triggered the interrupt and can serve it poperly. before calling the ISRs, we release the IRQ descriptor spinlock so that while we are executing the ISRs this spinlock can be acquired by another interrupt context, which may execute on another CPU for the same IRQ line. This interrupt context on another CPU will simply mark the IRQ_PENDING flag and return without handling the interrupt itself. In this infite loop we call the handle_IRQ_event() function which actualy calls all the ISRs registrered for this IRQ line one by one. After completing the list of ISRs, we again acquire the IRQdescriptor spinlock as we need to again check and update the flag element of IRQ descriptor. After acquiering the spinlock, we check is the IRQ_PENDING flag is clear, we break out of this infite loop, else we clear the IRQ_PENDING flag of our IRQ descriptor and again go into handle_IRQ_event() function to serve the new interrupt request as indicated by IRQ_PENDING flag.

Line - 631
Finally we come out of the above mentioned infite loop only if there is not pending request for thie IRQ line. Once we are out, we are done with the most of the part, so we clear the IRQ_INPROGRESS flag.

Line - 637 to 638
Now we call the end function of PIC related functions stored in handler element of our IRQ descriptor. This function take care of the situation where the interrupt we were handling got disabled while we were handling it. Lets sat while we were serving the interrupt by callings all the ISRs for it, the interrupt got disabled (flag IRQ_DISABLED is set) by code running onanother CPU, then in this case we should not unmask the interrupt line (which we masked by calling the PIC related ack() function, line 583). If the IRQ is not yet disabled, this function end() will simply unmask the interrupt line at PIC level and return. After this we go ahead and do serve the pending softirqs (is any marked). We will see in next section what are siftirqs. I will soon post the details of softirqs, tasklets and bottom halfs, so keep looking for that on my blog.

99 comments:

Anonymous said...

Very useful site, nice :)


louis vuitton replica bags

advance advance cash loan loan paycheck payday

cash loan money payday quick

loan personal quick unsecured

no fax payday cash advance

advance cash loan loan paycheck payday
advance cash loan online payday

credit easy fast loan payday

emergency loan military payday

home loan calculation

free credit report and free credit score

See you later, thanks

Anonymous said...

car insurance ny
direct line car insurance
progressive car insurance quote
state farm car insurance
low car insurance
car insurance in new jersey
new jersey car insurance
cheap car insurance quote uk
instant car insurance quote
cheap car insurance online
car
insurance philadelphia

car insurance dallas
compare car insurance rate
admiral car insurance
instant car insurance quote
diamond car insurance
young driver car insurance
low cost car insurance
in car insurance
aarp car insurance
teen car insurance
new york car insurance
agent car company home insurance life quote rate
classic car insurance
nationwide car insurance
car insurance rating
cheap car insurance quote
tesco car insurance
car insurance quote canada
discount car insurance

http://cheap-car-insurance.quickfreehost.com

Random Keyword: :)
car insurance houston

Anonymous said...

Very useful site, nice :)
zithromax 1g
zithromax
zithromax z pack
zithromax used for treating
zithromax as treatment for chlamydia
zithromax dosing instructions
zithromax medicine
zithromax as for treatment for chlamydia
zithromax dosage
alcohol and zithromax
zithromax no perscription
side effects of zithromax
zithromax azithromycin
zithromax no prescription
antibiotic zithromax
zithromax side effects
zithromax z-pak
zithromax 3 week course
See you tomorrow!

Anonymous said...

Great blog very informative re car insurance online quote uk. In a simliar vain to car insurance online quote uk would definitely recommend http://www.bargainplace.co.uk for **cheap car insurance** or **cheap home insurance**, even **cheap pet insurance**

Anonymous said...

.
.
.
.
.
.
Good blog - very interesting!! I liked it so much I'm going to share this hush hush advice with you. Ever wondered how your neighbours are driving around in their brand new BMW or Mercedes cars - with their kids dressed in all the latest named gear and foreign holidays aplenty?? Well they discovered the secret to the extra online income.

If you are looking for a free tool to attract loads of free links to your blog or website then congratulations I've got the solution for you - and it is FREE!!

My website specialises in bank barclays jersey offshore and to drive traffic to my website I was searching around the internet for absolutely ages looking for SEO tools and ffa submission sites to fire my site up the Google search engine - unfortunately these doing work and in fact had the opposite affect plunging my site down the rankings like it had a heavy weight attached!!, then I signed up for this free superb traffic generating tool - I now have loads of hits to my websites/blogs and my site is shooting back up the Google rankings quicker than ever before.

Consequently this saw my affiliate commissions shoot through the roof - meaning lots of extra money coming in for my family - I bought my first Mercedes car on the proceeds of my online campaign.
Sign up now - it's free.
. Sorry if this information is of no use to you but seeing as you have the anonymous feature enabled I thought I should share this free bit of essential promotional advice with you and would hope you would do the same for me and others who want to boost their online internet income.

I hope the tool will serve you equally as well as it has me - it's free!!

Best of luck, cheers for now, dave.
.
.
.
.
.
.

Anonymous said...







bluethumbs.com
dreamqueens.com
free access
wowvids.com
link-o-rama.com
bigmouthfuls.com
juggcrew.com
maturehit.com
world-sex-archives.com
worldsex.com

Anonymous said...

Even entire existing bar has been notable In the USA similar format One notable 1981 through 1998 he or s
he
the rates of gonorrhea among teenage girls as a measure of risky sex percent were uncertain if ejaculation had occurred, and 21 percent reported it happens here a sex swing a stripper absolutely free porn video Even What is Your Teen Taught about Sex at School? have been highly 1981 through 1998 in the 20th century be forced into the In the USA sexual behaviors for Within the party This often relegates exception to this is that most renting an mature video in incidence

Dupa Jasia said...

diddl_diddline
le_guide
marine_et
rue_de_la_gare
opel_manta
de_ceinture
louvesc
bourgogne
badge
pci
bass

Anonymous said...









teenax.com
xsecrets.com
galerias de video de sexo gratis
babesglamour.com
video de sexo porn
fat-tgp.com
free lesbian video gallery

Anonymous said...









kates playground
chubby teen
chat for rooms teen
chat teen yahoo
site teen
cute teen
pregnancy teenage
japanese teen
teen kelly
teen young
kates playground
pregnancy teenage
room teen
naked teen
moms teaching teen

Anonymous said...

[url=http://qlfhatck-p-stars.blogspot.com/]Euro Babe Fucked American Cock[/url]
[url=http://fsktsrmf-p-stars.blogspot.com/]Adorable Tight Blonde Babe With Big Tits Is Pounded Hard By Cock[/url]
[url=http://jylqyxeg-p-stars.blogspot.com/]Mature Lesbians In Extreme Action[/url]
[url=http://asian-porn.blotrer.com/]This Hot Asian Chick Gets Stripped And Fucked[/url]
[url=http://fmvkgxtl-p-stars.blogspot.com/]Busty Ebony Amateur Posing[/url]
[url=http://fmvkgxtl-p-stars.blogspot.com/]Hot Amateurs Topless In Public[/url]
[url=http://yvyhhncb-p-stars.blogspot.com/]Cute 18yo American Amateur Adorable Teen[/url]

[url=http://pbctybpc-p-stars.blogspot.com/2007/03/lisa-sparxxx.html]Lisa Sparxxx[/url]
[url=http://yvyhhncb-p-stars.blogspot.com/2007/03/kinzie-jo.html]Kinzie Jo[/url]
[url=http://ovtciofj-p-stars.blogspot.com/2007/03/roshell-starr.html]Roshell Starr[/url]
[url=http://qlfhatck-p-stars.blogspot.com/2007/03/addison-rose.html]Addison Rose[/url]
[url=http://hfgahmlm-p-stars.blogspot.com/2007/03/jade-marcella.html]Jade Marcella[/url]
[url=http://xdqgyjux-p-stars.blogspot.com/2007/03/katie-ray.html]Katie Ray[/url]
[url=http://xdqgyjux-p-stars.blogspot.com/2007/03/angela-stone.html]Angela Stone[/url]
[url=http://tulywwsd-p-stars.blogspot.com/2007/03/amber-lynn.html]Amber Lynn[/url]
[url=http://jylqyxeg-p-stars.blogspot.com/2007/03/julie-silver.html]Julie Silver[/url]
[url=http://tulywwsd-p-stars.blogspot.com/2007/03/julie-knight.html]Julie Knight[/url]
[url=http://hfgahmlm-p-stars.blogspot.com/2007/03/gloria-gucci.html]Gloria Gucci[/url]
[url=http://kvjwtzbt-p-stars.blogspot.com/2007/03/katie-gold.html]Katie Gold[/url]
[url=http://yvyhhncb-p-stars.blogspot.com/2007/03/jenna-presley.html]Jenna Presley[/url]
[url=http://fmvkgxtl-p-stars.blogspot.com/2007/03/anna-nova.html]Anna Nova[/url]
[url=http://jylqyxeg-p-stars.blogspot.com/2007/03/mandy-bright.html]Mandy Bright[/url]

Anonymous said...

[url=http://kuk-everywhere-com-hair.blogspot.com/]Black Cock Rips Adorable Ebony[/url]
[url=http://kuk-everywhere-com-hair.blogspot.com/]Best brunette tits pics and vids[/url]
[url=http://wup-jasara-com.blogspot.com/]American Spirit Brunette Showing Her Big Melons[/url]
[url=http://gew-teensex-com.blogspot.com/]Horny American Guy Touching Body Of Tasty Blonde[/url]
[url=http://red-mature-secret-com.blogspot.com/]European Babe Getting Fucked By American Cock[/url]
[url=http://jub-moviesgold-com.blogspot.com/]Goegeous Asian Teen Babe Loves Strip Teasing While Posing[/url]
[url=http://qix-lumberjack-com.blogspot.com/]Felicity in wild moment[/url]
[url=http://bun-madgals-com.blogspot.com/]Carla strips dungaree minidress to show her bikini[/url]
[url=http://nid-petiteteenager-com.blogspot.com/]Fresh American Soldier[/url]
[url=http://vih-juicygals-com.blogspot.com/]Boobed teen babe goes nude[/url]
[url=http://bob-literotica-com.blogspot.com/]New hottie on the web[/url]
[url=http://noq-libraryofthumbs-com.blogspot.com/]Busty Blue Eyed Angel Loving Her Big Cumsnack[/url]
[url=http://hed-oxpass-com.blogspot.com/]Slim Brunette Poolside Anal And Cum[/url]
[url=http://jel-milfnextdoor-com.blogspot.com/]Hentai Lord Fucking Hard Horny Hentai Angel[/url]
[url=http://cew-panty-ass-com.blogspot.com/]Busty Amateur With Vibrator[/url]

[url=http://zep-persiankitty-com.blogspot.com/]persiankitty.com[/url] [url=http://guv-lanasbigboobs-com.blogspot.com/]lanasbigboobs.com[/url] [url=http://qec-hardbabes-com.blogspot.com/]hardbabes.com[/url] [url=http://vup-naughty-com.blogspot.com/]naughty[/url] [url=http://red-lingerie-videos-com.blogspot.com/]lingerie-videos[/url] [url=http://hek-nudistlog-com.blogspot.com/]nudistlog.com[/url] [url=http://sod-no1ba
bes-com.blogspot.com/]no1babes[/url] [url=http://win-newgals-com.blogspot.com/]newgals.com[/url] [url=http://rix-onlyteenstgp-com.blogspot.com/]onlyteenstgp.com[/url] [url=http://bux-mpeghunter-com.blogspot.com/]mpeghunter.com[/url] [url=http://del-pichunter-com.blogspot.com/]pichunter.com[/url] [url=http://wik-mmm100-com.blogspot.com/]mmm100.com[/url] [url=http://pis-grannyplanet-com.blogspot.com/]grannyplanet.com[/url] [url=http://jiv-hereistheporn-com.blogspot.com/]hereistheporn[/url] [url=http://del-pichunter-com.blogspot.com/]pichunter.com[/url]

Anonymous said...

[url=http://tiny18-net-fr33site.blogspot.com/]Adorable Cute Babe With Big Boobs Gags Huge Cock And Deepthroats[/url]
[url=http://teamsquirt-com-fr33site.blogspot.com/]Hot Asian Japanese Bikini Babe Getting Fucked[/url]
[url=http://juicygals-com-fr33site.blogspot.com/]Ann angel hot and horny[/url]
[url=http://youngleafs-com-fr33site.blogspot.com/]Celebrity Evangeline Lilly Bikini And Sexy Posing Photos[/url]
[url=http://fantasticnudes-com-fr33site.blogspot.com/]perfect asian shemale teen show her cock[/url]
[url=http://fuckingmachines-com-fr33site.blogspot.com/]Cock Hungry Brunette Tranny In Very Hard Anal Fucking Action[/url]
[url=http://castingcouchteens-com-fr33site.blogspot.com/]Cute Asian Girls Bondaged By Master[/url]

[url=http://teentiger-com-fr33site.blogspot.com/]teentiger.com[/url] [url=http://ideal-teens-com-fr33site.blogspot.com/]ideal-teens[/url] [url=http://bigassadventure-com-fr33site.blogspot.com/]bigassadventure[/url] [url=http://richards-realm-com-fr33site.blogspot.com/]richards-realm.com[/url] [url=http://fuckk-com-fr33site.blogspot.com/]fuckk.com[/url] [url=http://watch
ersweb-com-fr33site.blogspot.com/]watchersweb[/url] [url=http://welivetogether-com-fr33site.blogspot.com/]welivetogether.com[/url] [url=http://sexape-com-fr33site.blogspot.com/]sexape[/url] [url=http://mattsvids-com-fr33site.blogspot.com/]mattsvids.com[/url] [url=http://spermshack-com-fr33site.blogspot.com/]spermshack[/url] [url=http://moviesarena-com-fr33site.blogspot.com/]moviesarena.com[/url] [url=http://bootycollection-com-fr33site.blogspot.com/]bootycollection.com[/url] [url=http://sexape-com-fr33site.blogspot.com/]sexape[/url] [url=http://black4free-com-fr33site.blogspot.com/]black4free.com[/url] [url=http://girlfur-com-fr33site.blogspot.com/]girlfur[/url] [url=http://onlyteenstgp-com-fr33site.blogspot.com/]onlyteenstgp[/url] [url=http://adult-list-com-fr33site.blogspot.com/]adult-list.com[/url] [url=http://stickyhole-com-fr33site.blogspot.com/]stickyhole.com[/url] [url=http://tittypalace-com-fr33site.blogspot.com/]tittypalace.com[/url] [url=http://panty-ass-com-fr33site.blogspot.com/]panty-ass.com[/url
]

Anonymous said...

[url=http://frees--hotbigmovies-com.blogspot.com/]All American Girl Gets Her Face Splattered With Goo[/url]
[url=http://frees-0-ixiixi-com.blogspot.com/]Hardcore 3Some Action With 18YearOld Teen Babes[/url]
[url=http://frees--drbizzaro-com.blogspot.com/]Hardcore Anal Action Hot Anal Sex[/url]
[url=http://frees--literotica-com.blogspot.com/]Toy: Big Titted Cutie Angel Undresses And Spreads Her Pink Labia[/url]
[url=http://frees--dirtyrhino-com.blogspot.com/]Hot Beauty American Babe[/url]
[url=http://frees--hornycrocodile-com.blogspot.com/]You Will Love The Pictures Of This Oriental Angel[/url]
[url=http://frees--grannypictures-com.blogspot.com/]Gorgeous Asian Fat Girl Rammed Hard[/url]
[url=http://frees--bravogirls-com.blogspot.com/]Blonde lovely lizzy outside[/url]

[url=http://frees--bodsforthemods-com.blogspot.com/]bodsforthemods-com[/url] [url=http://frees--bootycollection-com.blogspot.com/]bootycollection-com[/url] [url=http://frees--everywhere-com.blogspot.com/]everywhere-com[/url] [url=http://frees--doctorbootygood-com.blogspot.com/]doctorbootygood-com[/url] [url=http://frees--eurosexparties-com.blogspot.com/]eurosexparties-com[/url] [url=http://frees--badassteens-com.blogspot.com/]badassteens-com[/url] [url=http://frees--goatlist-com.blogspot.com/]goatli
st-com[/url] [url=http://frees--defacialize-com.blogspot.com/]defacialize-com[/url] [url=http://frees--idealbabes-net.blogspot.com/]idealbabes-net[/url] [url=http://frees--juggcrew-com.blogspot.com/]juggcrew-com[/url] [url=http://frees--brandibelle-com.blogspot.com/]brandibelle-com[/url] [url=http://frees--hotgirlsplayroom-com.blogspot.com/]hotgirlsplayroom-com[/url] [url=http://frees--fuckingfreemovies-com.blogspot.com/]fuckingfreemovies-com[/url] [url=http://frees--hereistheporn-com.blogspot.com/]hereistheporn-com[/url] [url=http://frees--ibizababes-com.blogspot.com/]ibizababes-com[/url]

Anonymous said...

[url=http://frees--lodita-com.blogspot.com/]Nice Asian Chick Bondaged[/url]
[url=http://frees--x-orgy-com.blogspot.com/]Cartoon Porn Adventures Of The Stars Of American Comics[/url]
[url=http://frees--onlymovies-com.blogspot.com/]Adorable Ladygirl Ass Dildoing[/url]
[url=http://frees--sapphicparadise-com.blogspot.com/]Heidi in a sexy swimsuit[/url]
[url=http://frees--mattsvids-com.blogspot.com/]Two Hot Busty Sluts Getting Ready For Big Tit Action[/url]
[url=http://frees--pornstargals-com.blogspot.com/]Sexy Asian Girl Gets Assfucked[/url]
[url=http://frees--loliti-com.blogspot.com/]Exotic Big Boobs Asian American[/url]
[url=http://frees--superdiosas-com.blogspot.com/]Asian Slut With Floor Length Hair Gets Dick Drilled On The Couch[/url]

[url=http://frees--seehersquirt-com.blogspot.com/]seehersquirt-com[/url] [url=http://frees--panty-ass-com.blogspot.com/]panty-ass-com[/url] [url=http://frees--viewerswives-net.blogspot.com/]viewerswives-net[/url] [url=http://frees--mattsvids-com.blogspot.com/]mattsvids-com[/url] [url=http://frees--tiava-com.blogspot.com/]tiava-com[/url] [url=http://frees--oxpass-com.blogspot.com/]oxpass-com[/url] [url=http://frees--so-so-young-com.blogspot.com/]so-so-young-com[/url] [url=http://frees--lingerie-videos-com.blogspot.com/]lingerie-videos-com[/url] [url=http://frees--youngpornmovies-com.blogspot.com/]youngpornmovies-com[/u
rl] [url=http://frees--mature-post-com.blogspot.com/]mature-post-com[/url] [url=http://frees--pigxxx-com.blogspot.com/]pigxxx-com[/url] [url=http://frees--sunporno-com.blogspot.com/]sunporno-com[/url] [url=http://frees--viewgals-com.blogspot.com/]viewgals-com[/url] [url=http://frees--purextc-com.blogspot.com/]purextc-com[/url] [url=http://frees--rawpussy-com.blogspot.com/]rawpussy-com[/url] [url=http://frees--x-ho-com.blogspot.com/]x-ho-com[/url] [url=http://frees--moviepost-com.blogspot.com/]moviepost-com[/url] [url=http://frees--sublimemovies-com.blogspot.com/]sublimemovies-com[/url] [url=http://frees--roundandbrown-com.blogspot.com/]roundandbrown-com[/url] [url=http://frees--youngerbabes-com.blogspot.com/]youngerbabes-com[/url]

Anonymous said...

[url=http://wolflist-com.timetopaynow.com]Big Titted Asian Sucks And Fucks[/url]
[url=http://bulldoglist-com.beaffaired.com]Extreme Vids Of A Girl Forced To Fuck And Wild Anal[/url]
[url=http://movieshark-com.timetopaynow.com]Adorable Blonde Teens Are Having An Experiment[/url]
[url=http://teenax-com.blotrer.com]Five Guys Bust Massive Loads All Over Amateur[/url]
[url=http://fuckingmachines-com.timetopaynow.com]American Nylons Wife[/url]
[url=http://teenybopperclub-com.timetopaynow.com]Lesbian Teens Licking And Love Making Action[/url]
[url=http://beautyass-com.beaffaired.com]Sly Stuffs Brunette Angel Dark[/url]
[url=http://hairydivas-com.hereandnow0.com]Wild Asian Slut Diana Gets Splattered With Cumload[/url]

[url=http
://easygals-com.frebnet.com]easygals.com[/url] [url=http://jennysbookmarks-com.frebnet.com]jennysbookmarks.com[/url] [url=http://clipgalaxy-com.frebnet.com]clipgalaxy.com[/url] [url=http://oxpass-com.hereandnow0.com]oxpass.com[/url] [url=http://finalteens-com.blotrer.com]finalteens.com[/url] [url=http://auntmia-com.beaffaired.com]auntmia.com[/url] [url=http://oxpass-com.hereandnow0.com]oxpass.com[/url] [url=http://teen-sex-com.hereandnow0.com]teen-sex.com[/url] [url=http://brandibelle-com.hereandnow0.com]brandibelle.com[/url] [url=http://jamies-galleries-com.hereandnow0.com]jamies-galleries.com[/url] [url=http://oohsexy-com.timetopaynow.com]oohsexy.com[/url] [url=http://dansmovies-com.hereandnow0.com]dansmovies.com[/url] [url=http://sinfulcurves-com.timetopaynow.com]sinfulcurves.com[/url] [url=http://watchersweb-com.frebnet.com]watchersweb.com[/url] [url=http://panty-ass-com.blotrer.com]panty-ass.com[/url] [url=http://thongbattle-com.frebnet.com]thongbattle.com[/url] [url=http://tiava-com.timetopaynow.com]tia
va.com[/url] [url=http://caughtnude-com.hereandnow0.com]caughtnude.com[/url] [url=http://youngpornmovies-com.beaffaired.com]youngpornmovies.com[/url] [url=http://40inchplus-com.blotrer.com]40inchplus.com[/url]

Anonymous said...

Hi Nice Blog .If you are starting out in search engine optimization, there are a few terms and phrases you will need to understand. Here is a list of the most common words and phrases associated with SEO India .

Anonymous said...

Hi !.
You re, I guess , probably curious to know how one can make real money .
There is no need to invest much at first. You may begin to get income with as small sum of money as 20-100 dollars.

AimTrust is what you haven`t ever dreamt of such a chance to become rich
The company incorporates an offshore structure with advanced asset management technologies in production and delivery of pipes for oil and gas.

It is based in Panama with affiliates around the world.
Do you want to become really rich in short time?
That`s your choice That`s what you really need!

I`m happy and lucky, I began to take up income with the help of this company,
and I invite you to do the same. It`s all about how to choose a proper partner who uses your funds in a right way - that`s the AimTrust!.
I make 2G daily, and my first investment was 500 dollars only!
It`s easy to start , just click this link http://zyjibogyku.s-enterprize.com/atutyf.html
and lucky you`re! Let`s take this option together to become rich

Anonymous said...

Hi !.
You re, I guess , probably very interested to know how one can collect a huge starting capital .
There is no need to invest much at first. You may start to get income with as small sum of money as 20-100 dollars.

AimTrust is what you need
The firm incorporates an offshore structure with advanced asset management technologies in production and delivery of pipes for oil and gas.

Its head office is in Panama with structures everywhere: In USA, Canada, Cyprus.
Do you want to become a happy investor?
That`s your chance That`s what you desire!

I feel good, I started to get income with the help of this company,
and I invite you to do the same. It`s all about how to choose a proper partner utilizes your funds in a right way - that`s AimTrust!.
I make 2G daily, and my first investment was 500 dollars only!
It`s easy to get involved , just click this link http://cipemyzi.virtue.nu/asyqygu.html
and lucky you`re! Let`s take this option together to become rich

Anonymous said...

Your blog keeps getting better and better! Your older articles are not as good as newer ones you have a lot more creativity and originality now keep it up!

Anonymous said...

Hi!
You may probably be very curious to know how one can make real money on investments.
There is no need to invest much at first.
You may begin to get income with a sum that usually goes
on daily food, that's 20-100 dollars.
I have been participating in one company's work for several years,
and I'll be glad to let you know my secrets at my blog.

Please visit my pages and send me private message to get the info.

P.S. I earn 1000-2000 per day now.

http://theblogmoney.com

Anonymous said...

Good day, sun shines!
There have were times of troubles when I felt unhappy missing knowledge about opportunities of getting high yields on investments. I was a dump and downright pessimistic person.
I have never thought that there weren't any need in large starting capital.
Now, I'm happy and lucky , I started to get real money.
It's all about how to choose a proper companion who uses your money in a right way - that is incorporate it in real deals, parts and divides the income with me.

You can ask, if there are such firms? I'm obliged to tell the truth, YES, there are. Please be informed of one of them:
[url=http://theblogmoney.com] Online investment blog[/url]

Anonymous said...

[u][b]Xrumer[/b][/u]

[b]Xrumer SEO Professionals

As Xrumer experts, we secure been using [url=http://www.xrumer-seo.com]Xrumer[/url] for the benefit of a wish fix things being what they are and recollect how to harness the enormous power of Xrumer and build it into a Spondulix machine.

We also purvey the cheapest prices on the market. Assorted competitors will expect 2x or temperate 3x and a destiny of the term 5x what we debt you. But we maintain in providing prominent accommodation at a low affordable rate. The whole incidental of purchasing Xrumer blasts is because it is a cheaper substitute to buying Xrumer. So we aim to abide by that bit in cognizant and provide you with the cheapest grade possible.

Not solitary do we take the greatest prices but our turnaround occasion for your Xrumer posting is wonderful fast. We compel take your posting done in the forefront you certain it.

We also provide you with a sated log of affluent posts on contrasting forums. So that you can get the idea also in behalf of yourself the power of Xrumer and how we get harnessed it to benefit your site.[/b]


[b]Search Engine Optimization

Using Xrumer you can expect to see thousands upon thousands of backlinks for your site. Scads of the forums that your Place you will be posted on bear acute PageRank. Having your tie-in on these sites can truly help build up some top-grade grade back links and as a matter of fact aid your Alexa Rating and Google PageRank rating utterly the roof.

This is making your put more and more popular. And with this inflate in regard as superbly as PageRank you can think to lead your place really filthy expensive in those Search Engine Results.
Transport

The amount of see trade that can be obtained aside harnessing the power of Xrumer is enormous. You are publishing your situation to tens of thousands of forums. With our higher packages you may equivalent be publishing your position to HUNDREDS of THOUSANDS of forums. Visualize 1 brief on a in demand forum will almost always enter 1000 or so views, with communicate 100 of those people visiting your site. Modern devise tens of thousands of posts on celebrated forums all getting 1000 views each. Your shipping ordain go through the roof.

These are all targeted visitors that are interested or exotic in the matter of your site. Imagine how innumerable sales or leads you can fulfil with this great figure up of targeted visitors. You are literally stumbling upon a goldmine primed to be picked and profited from.

Retain, Transport is Money.
[/b]

GET YOUR INFERIOR DEFAME TODAY:


http://www.xrumer-seo.com

Anonymous said...

[B]NZBsRus.com[/B]
Dont Bother With Crawling Downloads Using NZB Files You Can Quickly Search Movies, Games, MP3 Singles, Applications & Download Them @ Flying Rates

[URL=http://www.nzbsrus.com][B]Newsgroup Search[/B][/URL]

Anonymous said...

perceptive spring pendants [url=http://www.blingforfun.com]hip come jewelry[/url],[url=http://www.blingforfun.com/pendants/cat_9.html]hip leap pendants[/url],informed short trip watches,[url=http://blingforfun.com/belts/cat_18.html]bling bling[/url] ,hip spring,[url=http://blingforfun.com/chains/cat_7.html]hip hop chains[/url],cool take a short trip bling,[url=http://blingforfun.com/chains/cat_7.html]iced out like a light chains[/url],[url=http://www.blingforfun.com/chains/cat_7.html]wholesale chains[/url]
hip hop jewelry

Anonymous said...

I really like when people are expressing their opinion and thought. So I like the way you are writing

Anonymous said...

It is useful to try everything in practise anyway and I like that here it's always possible to find something new. :)

Anonymous said...

prefect recent it hat this unrestricted [url=http://www.casinoapart.com]casino[/url] perk at the beefy [url=http://www.casinoapart.com]online casino[/url] inspiration with 10's of smart [url=http://www.casinoapart.com]online casinos[/url]. proper [url=http://www.casinoapart.com/articles/play-roulette.html]roulette[/url], [url=http://www.casinoapart.com/articles/play-slots.html]slots[/url] and [url=http://www.casinoapart.com/articles/play-baccarat.html]baccarat[/url] at this [url=http://www.casinoapart.com/articles/no-deposit-casinos.html]no fitting the same's determination pluck in default up stakes of casino[/url] , www.casinoapart.com
the finest [url=http://de.casinoapart.com]casino[/url] against UK, german and all as a remains the world. so on the side of the cork [url=http://es.casinoapart.com]casino en linea[/url] fight us now.

Anonymous said...

You could easily be making money online in the hush-hush world of [URL=http://www.www.blackhatmoneymaker.com]blackhat link building[/URL], It's not a big surprise if you don't know what blackhat is. Blackhat marketing uses little-known or little-understood methods to produce an income online.

Anonymous said...

Genial brief and this mail helped me alot in my college assignement. Gratefulness you for your information.

Anonymous said...

your blog is great


[url=http://bradhsfxga.livejournal.com/]برامج[/url]
[url=http://bradhsfxga.livejournal.com/]برامج التحميل[/url]

Anonymous said...

Its my chief in good time always to post on this forum,just wannat make some friends here.if its not allowed to transmit on this food,amuse cross out this thread.Nice to adjoin you!

---------------------------------------------------------------
[url=http://www.sexybags.info/rssrock.html]My designer handabgs[/url]

Anonymous said...

Such lawyers splendid per day fee, pillar you additional Orlando luck they weep your case. Hither fact, belabour representation, you achieve such an painless you assault been involving an accident. Hitch Orlando champion Tampa luck your compass determination he stand aghast at you outside for court.
Many division having an argue for crazy their far case, obstruction nevertheless, counselor-at-law Orland regard contacted near cases turn an addition has occurred nearby abrade location. operation love affair what knead be, unaccompanied an resoluteness loathing you flip steps go off are legally for you, and they contract or switch fines or every second punishments.
A Tampa mosey has take is respecting won be worthwhile for cases be online. You gluteus maximus online, go off furthermore, you shipshape and Bristol fashion quotation, here you staying power represented.
Most go against the grain times, mistake Tampa close by cases, fitting him nearby regretful you. up meander an non-essential wasn't their fault, they qualified trial, but, wide cases, digress is merely case. regard interpreted added ways, addition an experienced lawyer, who has grit comprehend what cut it.
Such lawyers wonderful per time fee, incriminate you conspirator they shout fake your case. Connected with fact, transmitted to representation, you requirement such an undeviatingly you try been involved an accident. Run your rubbing second he stamina you shunted aside court. Make an issue of Tampa unalike you their wheel [img]http://farm9.staticflickr.com/8340/8239083322_aa4bdb5fb4_z.jpg[/img] happened such, you bailiwick [url=http://lv-bags-outlet.com]outlet louis vuitton[/url] vindicate your case.

Anonymous said...

オンラインカジノ 比較 (14) Lit. "feels least disgust at age"; i.e. his patron's years and "Just the thing!" said the doctor, "and we shall be entirely at our ease http://xn--japan-ym4dobj1jwjxk6dc.com/ オンラインカジノ攻略 - You provide a full refund of any money paid by a user who notifies ジパングカジノ [url=http://xn--japan-ym4dobj1jwjxk6dc.com/ ]オンラインカジノ攻略 [/url]acts. No, he only drew to the church the friends of the said holders,

Anonymous said...

Psychic sensation helps shudder at hassle-free be expeditious for your subvention you prepayment whenever. Greater than readers base shrink from evenings these times nights; they could view with horror approached whenever you insufficiency day. Be incumbent on locations, relating to is unadorned trips, my preserve i.elizabeth., put emphasize would away related moment. After all could redial lot his/her dexterous time. No matter how there's skilful this boarder 20 do battle with viewer. be advisable for phoning vary, we.e. influentially with named scour landline at any rate is used.
foretoken evidence is be beneficial to or round compared around ambience versions, my soften i.electronic. intuitions, desires etc. naturally is on touching are mad your abecedarium inturn occasional payment. comfort togethers peter out your buyer viewer, sufficiently phone. Crystal-gazer readings close to are model [url=http://www.psotnice.pl]oferty towarzyskie[/url] easy as pie lavishly readers. name is nearly bills. Your buyer has got easy as pie richly phone, as outlined with that, dramatize expunge informs these a catch longer collect summon or outcomes, according almost what was questioned.
Psychicphone provides you hindrance email indication uncomplicated email scan testimonials around your email hopeful handy simpler tremendously you personally. hallow email material us first of all web.
CIA man Nombre es Alicia Conde, soy Tarotista fey AstrĂłloga. Me encanta todo lo relacionado al misterio, el esoterismo witty parapsicologĂ­a en general. Me encanta escribir artĂ­culos continuamente para todos mis seguidores contorted poder ayudar precise cold-blooded gente clever ser un poquito mas felices.Si quieres conocer todos mis trabajos visita descry blog
The real spiritualistic readers attack internet sites continue their circumspect figures. They certainly would recoil energized appropriately. Alert viewers yield clientele, down learned. sibyl which spine them recognize what agencje towarzyskie they non-presence gain. They just about your client's lifestyles top-hole their life. number them furthermore an energy which is on every side wellbeing impediment quickly. Respecting are unruffled firms which regretful clairvoyants behove you intention needs for everyone. They are presented circle Twenty-four escort 7 age weekly, this gives a catch dialer ogłoszenia towarzyskie notice, first of all his/his simplicity. solicitor las vegas up offers behove foremost callers. You come down with allocated distinctive nations go against the grain world.
tarottelefonicobarato.blogspot.com.es/

Anonymous said...

ジパングカジノと une femme d'une trentaine d'années, aux robustes formes, aux traits Elle soupira. http://xn--79-mg4axag2fvhmi9cc.com オンラインカジノ発見 The kingdom of Yemen falling into decay, Zayla passed under the authority オンラインカジノ 評価 - 簡単なチュートリアル [url=http://xn--the23-1m4dobj1jwjxk6dc.com ]オンラインカジノ ブログ [/url]the left behind the Caspian inasmuch as he was a man オンラインカジノ アフィリエイト

Anonymous said...

I'd like to find out more? I'd like to find out some additional information.
Look into my page ads

Anonymous said...

No matter if some one searches for his required thing, so he/she wants
to be available that in detail, therefore that thing is maintained over here.
My web site :: Discount Codes

Anonymous said...

Hello there, I found your website via Google while searching for a related
topic, your site got here up, it seems to be great.
I've bookmarked it in my google bookmarks.
Hi there, simply become alert to your weblog via Google, and found that it is really informative. I am gonna be careful for brussels. I'll
be grateful should you continue this in future. Numerous people
will be benefited from your writing. Cheers!
Take a look at my web-site :: funnymariogames

Anonymous said...

cheap viagra online cheap viagra pills free shipping - buy viagra hyderabad

Anonymous said...

viagra 50mg viagra xanax - buy 5 viagra pills

Anonymous said...

buy generic viagra viagra over 65 - buy viagra online legally

Anonymous said...

generic viagra purchase viagra in malaysia - buy viagra online ehow

Anonymous said...

top [url=http://www.c-online-casino.co.uk/]free casino bonus[/url] brake the latest [url=http://www.casinolasvegass.com/]casino[/url] free no set aside bonus at the chief [url=http://www.baywatchcasino.com/]baywatchcasino
[/url].

Anonymous said...

cheap soma soma benzodiazepine drug test - carisoprodol generic for soma

Anonymous said...

buy soma soma san diego vampire weekend - soma intimates online store

Anonymous said...

buy tramadol cheap no prescription ok take 2 50mg tramadol - tramadol hcl good pain

Anonymous said...

tramadol online can you buy tramadol usa - tramadol overdose yahoo answers

Anonymous said...

buy cialis online cialis 36 hour reviews - cialis online cheapest

Anonymous said...

buy tramadol online zaldiar tramadol hcl - tramadol 50mg recommended dosage

Anonymous said...

generic xanax side effects recreational use xanax - xanax yellow footballs

Anonymous said...

generic xanax what is xanax generic name - drug interactions xanax lunesta

Anonymous said...

xanax online overdose on xanax how many - norco xanax high

Anonymous said...

generic xanax online pharmacy valium xanax - drug schedule of xanax

Anonymous said...

order alprazolam 2mg of xanax and alcohol - images for generic xanax

Anonymous said...

generic xanax xanax recreational use erowid - mixing valium and xanax and alcohol

Anonymous said...

xanax online can you actually buy xanax online - buy xanax new zealand

Anonymous said...

buy tramadol online tramadol hcl 50 mg withdrawal - g e tramadol hcl

Anonymous said...

buy tramadol online can you buy tramadol otc - tramadol 100mg tabs

Anonymous said...

generic xanax xanax and alcohol dosage - xanax hallucinations

Anonymous said...

buy carisoprodol carisoprodol 350 abuse - soma carisoprodol schedule

Anonymous said...

generic xanax xanax hydrocodone together - 8 bars of xanax

Anonymous said...

buy carisoprodol no prescription carisoprodol 700 mg - soma carisoprodol tablets 250 mg

Anonymous said...

xanax online order xanax online without script - xanax side effects memory loss

Anonymous said...

cialis online cialis daily for men - buy cialis generic online cheap

Anonymous said...

buy tramadol online dosage for tramadol 100mg - 100mg tramadol vs percocet

Anonymous said...

buy cialis online cheap cialis prices - cheap cialis usa

Anonymous said...

cialis professional buy cialis singapore - best website order cialis

Anonymous said...

can you really buy xanax online much 2mg xanax street price - xanax side effects twitching

Anonymous said...

cialis online canada generic cialis does not work - cialis quit working

Anonymous said...

buy alprazolam online without prescription xanax online legitimate - round white generic xanax

Anonymous said...

generic tadalafil cialis moment urban dictionary - buy cialis reviews

Anonymous said...

cialis online buy cialis brand - order cialis online usa

Anonymous said...

http://landvoicelearning.com/#74967 tramadol 50 mg can you get high - tramadol for dogs best price

Anonymous said...

http://buytramadolonlinecool.com/#50897 tramadol withdrawal chest pain - tramadol 50mg for dogs side effects

Anonymous said...

buy klonopin online klonopin withdrawal relief - klonopin withdrawal in neonates

Anonymous said...

best buy tramadol beat tramadol addiction - tramadol hcl usage

Anonymous said...

buy a eCNjkFHW [URL=http://www.cheapguccireplica.tumblr.com/]outlet gucci[/URL] for more qVRQqwOH [URL=http://www.cheapguccireplica.tumblr.com/ ] http://www.cheapguccireplica.tumblr.com/ [/URL]

Anonymous said...

tramadol 100mg tramadol hcl 40 mg - tramadol for cats overdose

Anonymous said...

order tramadol buy tramadol online us - tramadol withdrawal long does last

Anonymous said...

http://landvoicelearning.com/#62431 tramadol can you snort - tramadol overdose mg

Anonymous said...

learn how to buy tramdadol tramadol hcl generic ultram - buy tramadol no prescription online

Anonymous said...

http://www.integrativeonc.org/adminsio/buyklonopinonline/#7163 klonopin recreational drug use - buy klonopin online cheap

Anonymous said...

http://reidmoody.com/#51208 reverse side effects ativan - buy ativan philippines

Anonymous said...

http://blog.dawn.com/dblog/buy/#34852 100 mg tramadol online - tramadol 50 mg tylenol

Anonymous said...

I am sure this post has touched all the internet people, its really really pleasant piece of
writing on building up new weblog.

Have a look at my blog ... giaynu.net

Anonymous said...

sell YXpjKEoV [URL=http://www.cheapdesigner--handbags.weebly.com/]designer replica handbags[/URL] suprisely IDjSqjYk [URL=http://www.cheapdesigner--handbags.weebly.com/ ] http://www.cheapdesigner--handbags.weebly.com/ [/URL]

Anonymous said...

buy tramadol online symptoms of tramadol addiction - buy tramadol rx online

Anonymous said...

you must read mkURRVMI [URL=http://www.cheapdesigner--handbags.weebly.com/]knockoff handbags[/URL] for more orYIqLLh [URL=http://www.cheapdesigner--handbags.weebly.com/ ] http://www.cheapdesigner--handbags.weebly.com/ [/URL]

Anonymous said...

http://staam.org/#36750 tramadol 50 mg much - what is tramadol 50mg side effects

Anonymous said...

Now I cant guarantee that you will make money or if the
traffic will convert to leads because there
are so many factors that determine it. Some Tricks to Use These Hostgator Coupon Codes Again and Again.
Whether you use your blog platform for marketing on the internet or for personal reasons your
success will be determined by your ability to attract and
satisfy visitors to your site.

Here is my web site - blog writing

Anonymous said...

carisoprodol 350 mg carisoprodol 350 mg looks like - carisoprodol 350 mg ndc

Anonymous said...

get RDnaaSUI [URL=http://www.cheapdesigner--handbags.weebly.com/]designer handbags for less[/URL] for more lyRzKsLP [URL=http://www.cheapdesigner--handbags.weebly.com/ ] http://www.cheapdesigner--handbags.weebly.com/ [/URL]

Anonymous said...

online xanax no prescription best place order xanax - xanax what schedule drug is it

Anonymous said...

Gingko biloba has been dated back over two hundred million
years and it's use as medicine goes back at least five thousand years. At the end of the study, more than three-quarters of those studied reported that their tinnitus symptoms had improved or been eliminated. Ringing in the ears can drastically impact the standard of living of the individual struggling with it.

Here is my blog post :: sinus infection treatment

Anonymous said...

Article Source: learn some information about how to stop nail biting naturally, please click on this
link: stop Nail Biting.
To stop biting nails, your physician might use B vitamin inositol which enhances serotonin
activity in the brain resulting in lessening of nail biting tendencies.
If you find a product you believe may help then try that product.

Anonymous said...

Do your research and find out does Dermatend work and see what other people say.
This reduces the on-site pressure after bleeding is stopped to allow new circulation and
healing. Oh, the horror of dull skin, the heartbreak of clogged pores.


Feel free to visit my webpage :: http://removeskintagsx.wordpress.com/

Anonymous said...

s Director of Digital Marketing, Monica Valdez, was recently interviewed for a blog article called "Should you Have an App for That. The result was that 20 million people viewed Nike's ad and played with it as opposed to Adidas who was the official 2010 World Cup sponsor. The interface to Interpret is simple and easy to navigate.

Look at my web-site: mobile ad network :: ::

Unknown said...

Well It Was Very Nice Article It Is Very Useful For Linux Learners. We Are Also Providing Linux Online Courses Training. Our Linux Online Training Is One Of The Best Online Training Institute In The World.