Realtime capable Linux Xenomai by alicejenny


									  Realtime capable Linux: Requirements

                                              Application program     100% source compatibility
                                                                      better binary compatibility


         Realtime kernel                        Realtime Linux                      (Standard)Linux

       Response time: < 10 µs                                                     Response time: ~1ms
        One (physical)                                                       One process address space
   address space with n-tasks                                                      with n-threads

            All tasks have                                                   All threads of a process have
          one common API                     Response time: < 10 µs                 one common API
           implemented by                           same                             implemented by
         one common library                      homogeneity                       one common library

ASQF Fachgruppe Automation/ Automation Day                                                                   1
  Realtime capable Linux: The important questions

      • Is the POSIX-API suitable?
             – Only a subset is relevant for realtime applications
             – The mechanism for the notification of asynchronous events
               determine widespread usage
      • Can it provide for competitive performance?
             – Ideally it should be as good as specialized realtime kernels
             – Ultimately cycle times of 31,25 µs should be possible
      • What about additional changes to the Linux kernel ?
             – Ideally none
             – Only minimal changes allow for long time support

ASQF Fachgruppe Automation/ Automation Day                                    2
  The POSIX-API: Minimal set for realtime tasks
                   Thread management
                          – Creation,Termination, Scheduling
                   Synchronization
                          – Mutex
                          – Spinlock
                   Communication
                          – POSIX Message Queue
                          – Semaphore
                          – Shared Memory
                   Timer
                          – types (periodic, one shot, absolute, relative)
                          – resolution
                          – clock tick
                          – notification            needs improvements

ASQF Fachgruppe Automation/ Automation Day                                   3
  The POSIX Notification-API: The Timer-API
  timer_create (clockid_t clockid, timer_t *timerid, struct sigevent *evp);

                   CLOCK_REALTIME                     sigevent      SIGEV_NONE
                                             Timer                  SIGEV_SIGNAL

        The implementation within the glibc is not realtime capable
         another implementation fixes it
            (deterministic, performance improvement by a factor of 10)

        Sending a signal to the entire process is not deterministic. Potentially
        the entire thread list of a process must be scanned.
         minimal overhead and deterministic behavior require sending to a thread (Linux can handle it)
         no response based on signal handlers (signal handlers are for synchronous exceptions only)
         having explicit wait only allows for extremely fast notification.

ASQF Fachgruppe Automation/ Automation Day                                                                4
   The POSIX Notification-API: Extensions(1)
   An additional call sigevent_set_notification() makes sending the signal to a
   thread available at the API level. This call
      maps the threadid of the API to the tid of the kernel
      allows via an additional encoding of the sigev_notify field for
             supervision of timeliness
             implementation with best possible deterministic behavior

                                                                              POSIX definition
                                                                              (not very helpful)

                                      This situation must get            timer      Can be queried
                                      signalled                          overrun
       Scenario c
       Scenario b
       Scenario a

              timer fires                        timer fires                  timer fires

    The existing sigevent structure needs no changes.

ASQF Fachgruppe Automation/ Automation Day                                                             6
  The POSIX Notification-API: Extensions(2)
     A device must be a possible source of a timer tick (clock)
      Example: A Profinet Realtime Ethernet Controller ERTEC provides for a 250 µsec tick,
      not necessarily synchronous with CLOCK_REALTIME
        additional call register_clock()

     A firing timer is one special type of an event
      arbitrary asynchronous IO events should get signalled the same way
        additional call event_create()

                                     Timer                               SIGEV_SIGNAL (inhibited)

                         clock                  sigevent
                                     Timer                               SIGEV_THREAD (realtime capable
                                                                         implementation of the library module)
             Driver    register_clock()

                         direct IO-Event
                                                                         Signal to thread
                                                                         (optional with supervision)
                         create_event()               sigevent_set_notification()

                  Extensions to the POSIX API

ASQF Fachgruppe Automation/ Automation Day                                                                       7
  Realtime capable Linux: Base concept

                                                                                             Linux + Realtime
                                                              Process                       realtime Linux
                      Ipipe (ADEOS)          POSIX API                           Linux      kernel
                                                                                 99                          99

                                                         threads controlled by
                          realtime                       realtime scheduler
      HW-                 kernel
      Interrupts                                                                            boundary
                          Linux                          threads controlled
                          kernel                         by Linux-Scheduler
                                                                                 0                           0
                                                                                       static priorities
                                                Common Library                        (POSIX-counting)
               system call entry
               exactly the same for
               both kernels

ASQF Fachgruppe Automation/ Automation Day                                                                        8
  Realtime capable Linux: Functionality of the realtime kernel
                           Typical partitioning of a realtime thread
                   setup phase               event    time critical response path

                                                     Wait for event

     Much functionality needed but neither           minimal functionality needed
     • realtime requirements nor
     • high performance requirements                 minimal realtime kernel
                                                     - timer functions and its notification
     For all calls not implemented, the              - communication and synchronization
     thread temporarily runs under the               - scheduling
     control of the Linux scheduler

ASQF Fachgruppe Automation/ Automation Day                                                    9
  Realtime capable Linux: Limits of homogeneity
                   Signal source eg. timer                                   Linux process
     Ipipe                                              threads managed
                                                        by the
                                                        realtime scheduler
      HW-                    Realtime Kernel
                             Linux Kernel
                                                        threads managed
      Soft-                                             by the
      Interrupts                                        Linux scheduler

              Signal source eg. IO event

   A signal can only get sent to that domain which created the signal source
   This restriction is
         important for best possible performance (allows for separation of implementation)
         acceptable and manageable (violations are reported)

ASQF Fachgruppe Automation/ Automation Day                                                    10
  Realtime capable Linux: Implementation Kernel
 Two additional loadable modules:
 •     A device driver
         – functional extensions of the API (API-calls  ioctl() system calls,
           additional internal interfaces for realtime capable device drivers)
         – Interface for initialization/configuration of the realtime module
           eg. registration of a process as realtime process
 •     The realtime module (20K for x86)
         – Functionality of the realtime kernel
         – Communication between realtime and Linux kernel

 Three modified kernel files:
         – futex.c (hook), posix_timers.c (clock registration), mqueue.c

 Ipipe modifications:
         – Systemcall handling (additional hook at system call exit + optimization)
         – Interrupt handling (optimization, support for truly preemptive realtime kernel)

ASQF Fachgruppe Automation/ Automation Day                                                   11
  Realtime capable Linux: Implementation glibc

 Modified functions:
 •     Initialization
         – Registration of the realtime process
 •     Spinlock function also for static priorities
 •     Realtime capable implementation for SIGEV_THREAD

 Additional functions
 •     Device driver functions via ioctls
        – Registration of a timer clocks
        – Registration of an (IO-)event
 •     Thread specific sigevent notification (the kernel is able to do it)
 •     fast message queues

ASQF Fachgruppe Automation/ Automation Day                                   12
  Realtime capable Linux: System call overhead in clocks
                                                                        Cel. 2,8 GHz          Athlon 2 GHz
                                                                      1000 clocks = 357 ns   1000 clocks = 500 ns
                  Linux original                                 987/494                 343/147
                            Realtime Linux Domain

                                                    ipipe original       1155/641                     /185
      Linux + Ipipe patch

                                                    + AuD patch         1397/914                  466/338

                                                    + AuD optimized     1124/609                  383/257

                                                    + AuD patch         1260/767                  317/255

                                                    + AuD optimized      987/494                  289/199
           • Measurement with getuid(), since getpid() is handled inside glibc
           • Measurement with time stamp counter (94 clocks resp. 12/5 clocks)
           • glibc-2.3.6 generated with either int 80 or sysenter system entry (…/…)

ASQF Fachgruppe Automation/ Automation Day                                                                          13
  Realtime capable Linux: Performance inter-thread communication
                                                                            Two threads communicating either
                                                                            • via semaphores or
                                                                            • through POSIX message queues
                                 high priority thread
                     down                                          up


                                          sem_post()/mq_send()          Note:
                                                                        Times for down are always longer
                                     low priority thread                than for up, since an additional
                                                                        sem_wait() / mq_receive()-call is
        helper thread of                                                included.
         lowest priority

     Low priority thread must have
        a chance to proceed to
       sem_wait() / mq_receive()

ASQF Fachgruppe Automation/ Automation Day                                                                     14
  Realtime capable Linux: Interthread communication in clocks
                                               Celeron 2,8 GHz                       Athlon 64 2,0 GHz
                                             1000 clocks = 357 ns                    1000 clocks = 500 ns

                                       Semaphore       Message Queue          Semaphore      Message Queue
                                       up     down       up   down            up     down      up   down
                up                    2615     4547      5534       8106      836     1417     2050    2771
                      Realtime        2268     3980      4715       7004       744    1279     1963    2509
                                       3361    13304     7130    18575       1297     4943     2827    6666
  down          up                     3801    15992     7266    19825       1071     4374     2530    6174

                     Domain            3780     5377     7655    10364       1647     2215     3335    4197
                up                     3539     4715     6721       8390     1519     2075     3241    3912

              optimized                        Times for Realtime  Linux
                                               • first switch to interrupted Linux thread
    System call entry: sysenter                • afterwards switch to woken up Linux thread (message queue)
                                               • semaphores (futex) need an additional proxy thread for futex_wake

ASQF Fachgruppe Automation/ Automation Day                                                                           16
  Realtime capable Linux: Interrupt response time in clocks
                Celeron 2,8 GHz Athlon64 2,0 GHz                    Test scenario:
                 1000 clocks = 357 ns        1000 clocks = 500 ns   • An application thread issues via a driver an
                    min        max            min        max          ipipe soft interrupt
                     525         3276         224       1382        • The ISR resumes via send_event another
                                                                      application thread (waiting with sigtimedwait())
    response         284         1785         101       1024
                                                                    • ISR frequency varied between
     thread         3287       17441         1264       9240          100 µsec and 1 sec
     response                  16149          1010      9446
                   2688                                             • System load generated via GUI
                                                                      (Eclipse-Start, file transfer, etc.)

   The minimal figures are of interest in several respect:
   • for high interrupt frequencies representative for the average and therefore the load
   • typical for path length, and therefore useful for scalability projections
       • What can be achieved with less powerful HW (e.g. SOC with mediocre memory interface )?
   • for a dedicated processor of a multicore system (private TLB, L1-Cache, L2-Cache) the guaranteed
    worst case time comes closer to the minimal figure.

ASQF Fachgruppe Automation/ Automation Day                                                                               17
  Realtime capable Linux: Positioning to RT_PREEMPT
 • The kernel implementation is transparent to application programs
       – An application binary for a dual domain system runs without any changes on a Linux
         only system and therefore also on a RT_PREEMPT system

 • Possible coexistence scenarios with RT_PREEMPT
       – RT_PREEMPT as an alternative if no extremely strong realtime requirements
       – An additional realtime domain
             • for extremely strong requirements for response time and thread communication,
             • if demonstration is required (proof)
                tiny kernel, short paths  large complex kernel, lengthy paths, nested PI mutexes, RCU locks

 • Commonalities with RT_PREEMPT
       – The same modifications of the glibc are needed.
       – The same kernel enhancements (registering a clock, realtime capable POSIX
         message queues) are needed

ASQF Fachgruppe Automation/ Automation Day                                                                     18
  Realtime capable Linux: Conclusion
   • The POSIX-Realtime-Extensions need some fine tuning
         – Minimal extensions to the API definition improve usability and allow for best
           possible performance
   • Realtime under Linux can be provided transparently
         – The performance figures are comparable to specialized realtime kernels
   • Ipipe is a sustainable base
         – Small optimizations improve system call handling and interrupt response

ASQF Fachgruppe Automation/ Automation Day                                                 19

To top