PTHREADS-WIN32 SNAPSHOT 2001-07-03 ---------------------------------- Web Site: http://sources.redhat.com/pthreads-win32/ FTP Site: ftp://sources.redhat.com/pub/pthreads-win32 Coordinator: Ross Johnson We are pleased to announce the availability of a new snapshot of Pthreads-win32, an Open Source Software implementation of the Threads component of the POSIX 1003.1c 1995 Standard for Microsoft's Win32 environment. Some functions from POSIX 1003.1b are also supported including semaphores. Other related functions include the set of read-write lock functions. Parts of the implementation also comply with the Open Group's Single Unix specification for compatibility with major Unix implementations and Linux. Pthreads-win32 is free software, distributed under the GNU Library General Public License (LGPL). Please see the 'Acknowledgements' section at the end of this announcement for the list of contributors. Changes since the last snapshot ------------------------------- ----------------------- Additions to Scheduling ----------------------- New routines: pthread_attr_setinheritsched() pthread_attr_getinheritsched() pthread_attr_setschedpolicy() pthread_attr_getschedpolicy() sched_setscheduler() sched_getscheduler() sched_rr_get_interval() Now defined: _POSIX_THREAD_PRIORITY_SCHEDULING These routines complete the set required for defining _POSIX_THREAD_PRIORITY_SCHEDULING. sched_setscheduler sched_getscheduler ------------------ These routines will require patching to work with UWIN and any other system that provides it's own pid_t. Our pthread.h conditionally defines pid_t as a DWORD, which is the type returned by GetCurrentProcessId(). The only supported policy is SCHED_OTHER, however, in order to provide semantic compatibility these routines verify the following: - that the process identified by pid exists; - that permission is granted to set or query the policy; The appropriate error is returned if either of these fail. On success, both routines return SCHED_OTHER. sched_rr_get_interval --------------------- Always returns -1 and sets errno to ENOTSUP. This is implemented as a macro. It returns ENOTSUP rather than ENOSYS because sched_get_priority_max() and sched_get_priority_min() are supported, but only for SCHED_OTHER. pthread_attr_setschedpolicy pthread_attr_getschedpolicy --------------------------- The only valid supported policy is SCHED_OTHER. Attempting to set other policies results in an ENOTSUP error. pthread_attr_setinheritsched pthread_attr_getinheritsched ---------------------------- The two possible values that can be set are PTHREAD_INHERIT_SCHED and PTHREAD_EXPLICIT_SCHED. Neither the POSIX standard nor the Single Unix Spec specifies which should be the default value. Consequently, implementations use different defaults, eg (from a scan of the web): PTHREAD_INHERIT_SCHED default: HP, MKS Toolkit, QNX, AIX (?) PTHREAD_EXPLICIT_SCHED default: IBM OS/400, Linux, Sun All Win32 threads are created with THREAD_PRIORITY_NORMAL. They do not inherit the priority of the parent thread or the process. This behaviour is equivalent to the following Pthreads defaults: Inheritance: PTHREAD_EXPLICIT_SCHED Priority: THREAD_PRIORITY_NORMAL These are also the defaults in pthreads-win32, and now reinforced by changes to the library which now actually use these values and routines. This choice maintains the notion that, from the Pthread point-of-view, Win32 threads (those not created via pthread_create()) are treated as detached POSIX threads with default attribute values. ------------------ Changes to Mutexes ------------------ Background: Snapshot-2001-06-06 included Thomas Pfaff's enhancements to the mutex routines to improve speed. The improvements are very large on Win9x class systems where pthreads-win32 previously used Win32 mutexes rather than critical sections as the underlying mechanism. Based on some new benchtest programs that have been run on a Win98 machine (and included in the tests directory), WNT systems speed has probably decreased a little. On Win9x the enhancements also resulted in speed improvements in other primitives which use mutexes internally, such as condition variables and read-write locks. Thomas also added mutex types to the library as described in the Single Unix Specification documentation, and as provided with the majority of major Unix and Linux Pthreads implementations. Overall, the library now provides far more consistent performance across the different Windows variants and greater compatibility. Future work will continue to improve on this. New changes: Changes have been made to further improve the speed of the default PTHREAD_MUTEX_NORMAL type (and therefore also PTHREAD_MUTEX_DEFAULT which is equivalent in pthreads-win32). Specifically, the library no longer sets or checks the real mutex owner when locking, unlocking, trylocking, or destroying PTHREAD_MUTEX_NORMAL mutexes. This saves significant overhead and results in measured speed increases of around 90 percent for non-blocking lock operations, and a slight improvement for blocking lock operations. Since the type of mutex used internally is PTHREAD_MUTEX_DEFAULT, this also results in additional speed improvements to CVs and R/W lock operations. Subjective observation shows an improvement of up to 30-35% in R/W locks (from tests/rwlock7.c). This is compared to the already improved snapshot-2001-06-06. The price paid for this improvement is that some checking for errors is not done for these mutex types. The onus is placed on the developer to check application code for logical errors, or use a safer mutex type such as PTHREAD_MUTEX_ERRORCHECK. For example, it is now possible for a non-owner thread to unlock or destroy a mutex of these types. However, because the owner is not simply left as NULL but set to a special anonymous owner value when locked and then reset to NULL when unlocked or destroyed, an error will ultimately eventuate when the owner thread subsequently attempts to unlock or destroy the mutex. These changes are consistent with both the behaviour exhibited by PTHREAD_MUTEX_NORMAL in other implementations and their documentation, including the Open Group documentation. ------------ Benchmarking ------------ There is a new but growing set a benchmarking programs in the "tests" directory. These should be runnable using the following command-lines corresponding to each of the possible library builds: MSVC: nmake clean VC-bench nmake clean VCE-bench nmake clean VSE-bench Mingw32: make clean GC-bench make clean GCE-bench UWIN: The benchtests are run as part of the testsuite. In this snapshot there are four benchtests timing various mutex senarios. They are: benchtest1 - Lock plus unlock on an unlocked mutex. benchtest2 - Lock plus unlock on a locked mutex. benchtest3 - Trylock on a locked mutex. benchtest4 - Trylock plus unlock on an unlocked mutex. Each test times up to three alternate synchronisation implementations as a reference, and then times each of the four mutex types provided by the library. Each is described below: Simple Critical Section - uses a simple Win32 critical section. There is no additional overhead for this case as there is in the remaining cases. POSIX mutex implemented using a Critical Section - The old implementation which uses runtime adaptation depending on the Windows variant being run on. When the pthreads DLL was run on WinNT or higher then POSIX mutexes would use Win32 Critical Sections. POSIX mutex implemented using a Win32 Mutex - The old implementation which uses runtime adaptation depending on the Windows variant being run on. When the pthreads DLL was run on Win9x then POSIX mutexes would use Win32 Mutexes (because TryEnterCriticalSection is not implemented on Win9x). PTHREAD_MUTEX_DEFAULT PTHREAD_MUTEX_NORMAL PTHREAD_MUTEX_ERRORCHECK PTHREAD_MUTEX_RECURSIVE - The current implementation supports these mutex types. The underlying basis of POSIX mutexes is now the same irrespective of the Windows variant. In all benchtests, the operation is repeated a large number of times and an average is calculated. Loop overhead is measured and subtracted from all test times. --------- Bug fixes --------- Pthread_create now sets the priority of the new thread from the value set in the thread attribute. - from Ralf.Brese@pdb4.siemens.de. In the last snapshot I introduced a "lost signal" bug into the condition variables code. - fixed by Alexander Terekhov - reported by Timur Aydin taydin@snet.net --------- New tests --------- Several new tests have been added to the test suite. --------------------------- Known bugs in this snapshot --------------------------- 1. Under MS VC++ (only tested with version 6.0), a term_func set via the standard C++ set_terminate() function is not called for some reason. The code - in private.c:ptw32_threadStart() - makes an explicit call to terminate() which works as expected under MinGW32 g++ doesn't appear to run the term_func under MC VC++ 6.0. 2. Cancellation problems in optimised code - Milan Gardian The cancellation (actually, cleanup-after-cancel) tests fail when using VC (professional) optimisation switches (/O1 or /O2) in pthreads library. I have not investigated which concrete optimisation technique causes this problem (/Og, /Oi, /Ot, /Oy, /Ob1, /Gs, /Gf, /Gy, etc.), but here is a summary of builds and corresponding failures: * pthreads VSE (optimised tests): OK * pthreads VCE (optimised tests): Failed "cleanup1" test (runtime) * pthreads VSE (DLL in CRT, optimised tests): OK * pthreads VCE (DLL in CRT, optimised tests): Failed "cleanup1" test (runtime) Please note that while in VSE version of the pthreads library the optimisation does not really have any impact on the tests (they pass OK), in VCE version addition of optimisation (/O2 in this case) causes the tests to fail uniformly - either in "cleanup0" or "cleanup1" test cases. Please note that all the tests above use default pthreads DLL (no optimisations, linked with either static or DLL CRT, based on test type). Therefore the problem lies not within the pthreads DLL but within the compiled client code (the application using pthreads -> involvement of "pthread.h"). I think the message of this section is that usage of VCE version of pthreads in applications relying on cancellation/cleanup AND using optimisations for creation of production code is highly unreliable for the current version of the pthreads library. Caveats ------- 1. Due to what is believed to be a C++ compliance error in VC++, if your application contains catch(...) blocks in your POSIX threads then you will need to replace the "catch(...)" with the macro "PtW32Catch", eg. #ifdef PtW32Catch PtW32Catch { ... } #else catch(...) { ... } #endif Otherwise neither pthreads cancelation nor pthread_exit() will work reliably. Level of standards conformance ------------------------------ The following POSIX 1003.1c 1995 and POSIX 1003.1b options are defined: _POSIX_THREADS _POSIX_THREAD_SAFE_FUNCTIONS _POSIX_THREAD_ATTR_STACKSIZE _POSIX_THREAD_PRIORITY_SCHEDULING _POSIX_SEMAPHORES The following POSIX 1003.1c 1995 options are not defined: _POSIX_THREAD_ATTR_STACKADDR _POSIX_THREAD_PRIO_INHERIT _POSIX_THREAD_PRIO_PROTECT _POSIX_THREAD_PROCESS_SHARED The following functions are implemented: --------------------------- PThreads --------------------------- pthread_attr_init pthread_attr_destroy pthread_attr_getdetachstate pthread_attr_getstackaddr pthread_attr_getstacksize pthread_attr_setdetachstate pthread_attr_setstackaddr pthread_attr_setstacksize pthread_create pthread_detach pthread_equal pthread_exit pthread_join pthread_once pthread_self pthread_cancel pthread_cleanup_pop pthread_cleanup_push pthread_setcancelstate pthread_setcanceltype pthread_testcancel --------------------------- Thread Specific Data --------------------------- pthread_key_create pthread_key_delete pthread_setspecific pthread_getspecific --------------------------- Mutexes --------------------------- pthread_mutexattr_init pthread_mutexattr_destroy pthread_mutexattr_getpshared pthread_mutexattr_setpshared pthread_mutexattr_gettype pthread_mutexattr_settype (types: PTHREAD_MUTEX_DEFAULT PTHREAD_MUTEX_NORMAL PTHREAD_MUTEX_ERRORCHECK PTHREAD_MUTEX_RECURSIVE ) pthread_mutex_init pthread_mutex_destroy pthread_mutex_lock pthread_mutex_trylock pthread_mutex_unlock --------------------------- Condition Variables --------------------------- pthread_condattr_init pthread_condattr_destroy pthread_condattr_getpshared pthread_condattr_setpshared pthread_cond_init pthread_cond_destroy pthread_cond_wait pthread_cond_timedwait pthread_cond_signal pthread_cond_broadcast --------------------------- Read/Write Locks: --------------------------- pthread_rwlock_init pthread_rwlock_destroy pthread_rwlock_tryrdlock pthread_rwlock_trywrlock pthread_rwlock_rdlock pthread_rwlock_rwlock pthread_rwlock_unlock --------------------------- Semaphores --------------------------- sem_init (POSIX 1b) sem_destroy (POSIX 1b) sem_post (POSIX 1b) sem_wait (POSIX 1b) sem_trywait (POSIX 1b) sem_open (POSIX 1b - returns an error ENOSYS) sem_close (POSIX 1b - returns an error ENOSYS) sem_unlink (POSIX 1b - returns an error ENOSYS) sem_getvalue (POSIX 1b - returns an error ENOSYS) --------------------------- RealTime Scheduling --------------------------- pthread_attr_getschedparam pthread_attr_setschedparam pthread_attr_getinheritsched pthread_attr_setinheritsched pthread_attr_getschedpolicy (only supports SCHED_OTHER) pthread_attr_setschedpolicy (only supports SCHED_OTHER) pthread_getschedparam pthread_setschedparam pthread_getconcurrency pthread_setconcurrency pthread_attr_getscope (returns an error ENOSYS) pthread_attr_setscope (returns an error ENOSYS) sched_get_priority_max (POSIX 1b) sched_get_priority_min (POSIX 1b) sched_rr_get_interval (POSIX 1b - returns an error ENOTSUP) sched_setscheduler (POSIX 1b - only supports SCHED_OTHER) sched_getscheduler (POSIX 1b - only supports SCHED_OTHER) sched_yield (POSIX 1b) --------------------------- Signals --------------------------- pthread_sigmask --------------------------- Non-portable routines (see the README.NONPORTABLE file for usage) --------------------------- pthread_getw32threadhandle_np pthread_delay_np pthread_mutexattr_getkind_np pthread_mutexattr_setkind_np (types: PTHREAD_MUTEX_FAST_NP, PTHREAD_MUTEX_ERRORCHECK_NP, PTHREAD_MUTEX_RECURSIVE_NP) pthread_win32_process_attach_np pthread_win32_process_detach_np pthread_win32_thread_attach_np pthread_win32_thread_detach_np --------------------------- Static Initializers (macros) --------------------------- PTHREAD_ONCE_INIT PTHREAD_MUTEX_INITIALIZER PTHREAD_COND_INITIALIZER PTHREAD_RWLOCK_INITIALIZER --------------------------- Thread-Safe C Runtime Library (macros) --------------------------- strtok_r asctime_r ctime_r gmtime_r localtime_r rand_r The following functions are not implemented: --------------------------- RealTime Scheduling --------------------------- pthread_mutex_getprioceiling pthread_mutex_setprioceiling pthread_mutex_attr_getprioceiling pthread_mutex_attr_getprotocol pthread_mutex_attr_setprioceiling pthread_mutex_attr_setprotocol --------------------------- Fork Handlers --------------------------- pthread_atfork --------------------------- Stdio --------------------------- flockfile ftrylockfile funlockfile getc_unlocked getchar_unlocked putc_unlocked putchar_unlocked --------------------------- Thread-Safe C Runtime Library --------------------------- readdir_r getgrgid_r getgrnam_r getpwuid_r getpwnam_r --------------------------- Signals --------------------------- pthread_kill sigtimedwait sigwait sigwaitinfo The library includes two non-API functions for creating cancellation points in applications and libraries: pthreadCancelableWait pthreadCancelableTimedWait Availability ------------ The prebuilt DLL, export libs (for both MSVC and Mingw32), and the header files (pthread.h, semaphore.h, sched.h) are available along with the complete source code. The source code can be found at: ftp://sources.redhat.com/pub/pthreads-win32 and as individual source code files at ftp://sources.redhat.com/pub/pthreads-win32/source The pre-built DLL, export libraries and include files can be found at: ftp://sources.redhat.com/pub/pthreads-win32/dll-latest Mailing List ------------ There is a mailing list for discussing pthreads on Win32. To join, send email to: pthreads-win32-subscribe@sourceware.cygnus.com Application Development Environments ------------------------------------ See the README file for more information. MSVC: MSVC using SEH works. Distribute pthreadVSE.dll with your application. MSVC using C++ EH works. Distribute pthreadVCE.dll with your application. MSVC using C setjmp/longjmp works. Distribute pthreadVC.dll with your application. Mingw32: You need gcc-2.95.2-1 modified as per pthreads-win32 FAQ answer (6), with binutils-19990818-1 and msvcrt runtime-2000-03-27. Mingw32 must use the thread-safe MSVCRT library (see the FAQ). You need to distribute the gcc.dll DLL from Mingw32 with your application. Mingw using C++ EH works. Distribute pthreadGCE.dll with your application. Mingw using C setjmp/longjmp works. Distribute pthreadGC.dll with your application. Cygwin: (http://sourceware.cygnus.com/cygwin/) Cygwin aims to provide a complete POSIX environment on top of Win32, including threads. When this is complete, developers using Cygwin will not need pthreads-win32. At this time, Cygwin has preliminary support for multithreaded development, however, this is not turned on by default. We have not tested pthreads-win32 against Cygwin. UWIN: UWIN is a complete Unix-like environment for Windows from AT&T. Pthreads-win32 doesn't currently support UWIN (and vice versa), but that may change in the future. Generally: For convenience, the following pre-built files are available on the FTP site (see Availability above): pthread.h - for POSIX 1c threads semaphore.h - for POSIX 1b semaphores sched.h - for POSIX 1b scheduling pthreadVCE.dll - built with MSVC++ compiler using C++ EH pthreadVCE.lib pthreadVC.dll - built with MSVC compiler using C setjmp/longjmp pthreadVC.lib pthreadVSE.dll - built with MSVC compiler using SEH pthreadVSE.lib pthreadGCE.dll - built with Mingw32 G++ 2.95.2-1 pthreadGC.dll - built with Mingw32 GCC 2.95.2-1 using setjmp/longjmp libpthreadGCE.a - derived from pthreadGCE.dll libpthreadGC.a - derived from pthreadGC.dll gcc.dll - needed if distributing applications that use pthreadGCE.dll These are the only files you need in order to build POSIX threads applications for Win32 using either MSVC or Mingw32. See the FAQ file in the source tree for additional information. Documentation ------------- Currently, there is no documentation included in the package apart from the copious comments in the source code. For POSIX Thread API programming, several reference books are available: Programming with POSIX Threads David R. Butenhof Addison-Wesley (pub) Pthreads Programming By Bradford Nichols, Dick Buttlar & Jacqueline Proulx Farrell O'Reilly (pub) On the web: see the links at the bottom of the pthreads-win32 site: http://sources.redhat.com/pthreads-win32/ Acknowledgements ---------------- This library is based substantially on a Win32 pthreads implementation contributed by John Bossom . The implementation of Condition Variables uses algorithms developed by Alexander Terekhov and Louis Thomas. The implementation of POSIX mutexes has been improved by Thomas Pfaff. The implementation of read/write locks was contributed by Aurelio Medina and improved by Alexander Terekhov. Many others have contributed significant time and effort to solve critical problems in order to make the library workable, robust and reliable. There is also a separate CONTRIBUTORS file. This file and others are on the web site: http://sources.redhat.com/pthreads-win32 As much as possible, the ChangeLog file acknowledges contributions to the code base in more detail. Enjoy! Ross Johnson