Beta
×

Welcome to the Slashdot Beta site -- learn more here. Use the link in the footer or click here to return to the Classic version of Slashdot.

Thank you!

Before you choose to head back to the Classic look of the site, we'd appreciate it if you share your thoughts on the Beta; your feedback is what drives our ongoing development.

Beta is different and we value you taking the time to try it out. Please take a look at the changes we've made in Beta and  learn more about it. Thanks for reading, and for making the site better!

Why Isn't X11 Thread-Safe?

Cliff posted more than 11 years ago | from the correcting-a-potential-underlying-problem dept.

X 44

blackcoot asks: "I've just spent a couple very frustrating days trying to figure out what 'unexpected async reply' means and fixing it. The problem is a result of the fact that X11 simply isn't designed to handle events from more than one thread at a time. Why? Given that more and more often, people are writing multi-threaded GUI applications, are there fundamental design decisions in X11 that make dealing with receiving events from multiple threads simultaneously, impossible? Or was the protocol never designed to handle concurrent updates? More to the point, is there an easy way in Qt (short of deriving a new widget for every widget and overriding it's paintEvent to lock the library first, paint, then unlock as Trolltech's docs seem to suggest) to make this problem go away?" I'm not sure if things have been done in recent revisions of XFree to fix this problem, but this message, from February of last year, might help some of you out that are suffering from this problem. Any ideas if this problem has been fixed in recent versions of XFree?

cancel ×

44 comments

Sorry! There are no comments related to the filter you selected.

OFFICIAL FIRST POST DECLARATION NOTICE (2, Funny)

Anonymous Coward | more than 11 years ago | (#5097036)

Hello,

Pursuant to the universal Slashdot Act of First Postage, it is hereby posted that this post, article number referenced above, is a declaration for first post in the following circumstances:

1. This post claims first post in this article, persuant to the limitations outlined in 3(a).
2. This post declares under Slashdot Bylaw TROLL 13.22 that this first post shall be unique, and all other posts of similar nature shall be moderated as accorded by Slashdot Bylaw TROLL 13.22b
3a. According to the Slashdot Bylaws, this post recognizes that other posts have been made to this article.
3b. According to the Slashdot Bylaws, the current reader agrees to accept this post as the rightful first post.

don't use threads! (0, Insightful)

Anonymous Coward | more than 11 years ago | (#5097232)

threads are stupid except for computational work.

GUIs are naturally event driven, there is no reason to use threads at all. just use an event loop.

Re:don't use threads! (3, Insightful)

michaelggreer (612022) | more than 11 years ago | (#5097314)

True, if all the app does is GUI. Perhaps, though, when you fire events in the GUI, something occurs. Surely you would not want the GUI to wait on the result of that fired procedure: you would execute it in a seperate thread. Or, maybe the GUI is not entirely static (a movie clip is showing, for instance). You would not want the event updates fighting with the movie playback: you would run them in seperate threads.

Re:don't use threads! (3, Insightful)

Euphonious Coward (189818) | more than 10 years ago | (#5097705)

michaelggreer wrote: "You would not want the event updates fighting with the movie playback"

If some events should have higher priority than others, your GUI thread should schedule them properly. It would be a grave design error for the X server to have to know about threads in the client, and their relative priority. However, finding yourself implementing a priority scheduler in user-space is often an indication that you have made a wrong turn in your architecture. Scheduling is what OSes are for.

As an alternative, the client is free to open multiple connections to the X server, and operate independent UIs through them. As far as the X server is concerned, it's talking to independent clients, something it is very good at.

X sux and is the biggest thing holding back linux (-1, Redundant)

Anonymous Coward | more than 11 years ago | (#5097268)

It needs to be taken out the back and shot through the head.

Thank god that Mac OS X isn't using it.

Re:X sux and is the biggest thing holding back lin (1)

yandros (38911) | more than 10 years ago | (#5097977)

Like democracy, X is an absolutely terrible system, clearly the worst choice, except for every other choice we have so far.

Hey, there are projects out there that would like to replace X, so there's hope.. In the meantime, there are a number of areas where X is a better choice than the primary `alternatives' (Windows and Mac OS; distinct from OS, network-capable, configurable/modular, source available).

Re:X sux and is the biggest thing holding back lin (2, Interesting)

JohnFluxx (413620) | more than 10 years ago | (#5098497)

Why do you think X sucks?

It is a very impressive protocol - just consider how long it has been around, and you can do just about anything without breaking it.
How many other protocols are there that can boast that?

What can't X do?

Re:X sux and is the biggest thing holding back lin (0)

Anonymous Coward | more than 11 years ago | (#5099004)

I think the only people who have a problem with X are the people who have only experienced it on XFree/Linux.

Re:X sux and is the biggest thing holding back lin (1)

CptNoSkill (528594) | more than 11 years ago | (#5099988)

..What can't X do... Be combined with two other X servers, and become XXX and entertain me for hours..... Oh, you ment seriously... 4.3 seems to fix the command complaints... (Switching resolutions, if I'm not mistaken...)

Re:X sux and is the biggest thing holding back lin (1)

Black Parrot (19622) | more than 11 years ago | (#5099162)


> It needs to be taken out the back and shot

Let us know when your competing system is ready for us to test.

Re:X sux and is the biggest thing holding back lin (0)

Anonymous Coward | more than 11 years ago | (#5100237)

Mine's been ready for a while. My project's page is here [microsoft.com]

Take care,
Bill

Re:X sux and is the biggest thing holding back lin (2, Funny)

Gordonjcp (186804) | more than 11 years ago | (#5101235)

Hmmm, a framebuffer with network access crufted in on top? It'll never fly...

Re:X sux and is the biggest thing holding back lin (0)

Anonymous Coward | more than 11 years ago | (#5102692)

Hey, everybody! Black Parrot is a liar! He claimed that "Microsoft destroyed my company," but when challenged, he refused to post any facts to back that statement up. See this thread for all the gory details. Black Parrot is an anti-Microsoft zealot and a liar! Whenever he posts an unsubstantiated assertion, tell him to "post or retract" and watch the backpedaling begin! Don't let anything he says go unchallenged!

This message was brought to you by Trolls Aligned Against People Who Are Really Stupid (TAAPWARS).

What I do (5, Interesting)

Procyon101 (61366) | more than 11 years ago | (#5097426)

What I do is create a GUI thread which handles all interface to the GUI. All of my other threads then fire their events to that thread through an interface that IS threadsafe. The GUI thread then synchronously handles the GUI events. This abstraction away from your actual GUI API allows you to more easily port your app to another platform because all of your X11 specific code is contained in one place.

Swing does this (3, Informative)

Jack Greenbaum (7020) | more than 10 years ago | (#5098181)

This is the same approach taken by Swing ("lightweight" layer on top of Java AWT). Events fired by a GUI object are run in a GUI thread. For side effects from a non-GUI object, a convenience class is provide to push events into the GUI thread. Basically to make a GUI call from a non-GUI thread you throw a work request onto a queue which the Swing thread processes at an appropriate time. -- Jack

Old, old VB trick applies (0)

Anonymous Coward | more than 10 years ago | (#5098247)

Queue all window/asyncyronous events and have one thread processing them all.

Stupid widnows VB/vc++ programs would have to send themselves a queue-able event for any interrupt driven things so that it would not mess up any currently executing GUI related events.

Re:What I do (3, Interesting)

mike_sucks (55259) | more than 11 years ago | (#5098878)

Yeah, I think having a single GUI thread per app is pretty much standard for all the environments I've programmed for.

There's absolutely no need for a display system like X to allow multiple application threads to concurrently recieve events and/or update the application's display. It would just add unnesessary complexity, making it harder to debug and maintain.

Also, having a single GUI thread is a good design pattern. All GUI work is handled by one thead, all application logic is (potentially) handled by other threads. The delineation between the application's UI and logic makes it much easier to maintain.

So yeah, blackcoot shouldn't complain that X is broken, I'd say whatever app {s}he is writing needs to get fixed.

Multi-threading is GOOD [was Re:What I do] (3, Informative)

nellardo (68657) | more than 11 years ago | (#5100242)

There's absolutely no need for a display system like X to allow multiple application threads to concurrently recieve events and/or update the application's display.
First, you should restrict such broad generalizations to an X client in order to be even remotely correct. The X Server doesn't know anything about the process or thread structure of its clients. The clients may not even be on the same machine. The X Server gets a socket connection and responds to messages on that socket. If something is sending good messages down that socket, the X Server doesn't care if the something is one process, ten threads, or ten million threads.

Go look at KParts - KDE embedding. One window, one application, as far as the user is concerned, but subwindows are controlled possibly by different processes on different machines.

How do you think window managers work? The X server doesn't know from window managers - the way you prevent multiple window managers is by checking for atoms on the root window. Remember, conceptually any client of a server can do something with any window - you just need a way to get the window ID.

It would just add unnesessary complexity, making it harder to debug and maintain.
Wrong. The X server already handles multiple simultaneous connections. Whether the X client does or not is the client's choice. Lots of clients for other systems handle multiple connections (your web browser, for one).
Also, having a single GUI thread is a good design pattern.
Um, add the hedge for some applications and I'll agree with you. Otherwise you're wedged in a one-track design mind. You're making your problem fit your design, rather than your design fit your problem. There's any number of interactive applications that make lots of logical sense to be multi-threaded. Take your typical movie player with its nifty visual feedback. One thread for putting the frames up. One thread to respond to user actions.

The richer your interaction, the more application semantics are involved, the more likely that arbitrarily splitting the app in half along an arbitrary "UI/App" line is just not going to work. MVC has a similar problem. They're both cookie-cutter designs that pretend that every interactive app is structured the same way at the top level.

All GUI work is handled by one thead, all application logic is (potentially) handled by other threads. The delineation between the application's UI and logic makes it much easier to maintain.
Go read papers on the "eXene" system in the programming language "ML". A pervasively multi-threaded X client library. One widget - one thread. It makes the widget code very easy to understand - you don't have to split your code into a bazillion little callbacks. You don't have to arbitrarily time-slice things that are conceptually continuous. Things like while (buttonIsDown) followTheMouse() work just fine. Ever have to break up a callback into multiple functions, triggered by timers, just so the app didn't appear to "freeze" while you were off doing something time-intensive? Multi-threading interaction can make the code much easier to maintain because you don't have to worry about "starving" parts of the application for events while busy working on others - the thread scheduler handles pre-emption for you.

And eXene isn't some hot new thing. It dates from the early 90's.

Go back even earlier and find some of James "Java" Gosling's earliest work - NeWS. NeWS clients wrote multi-threaded PostScript to draw on the display.

Events, timer callbacks and the like are all just ways of simulating something continuous with discrete code - go look at TBAG from Sun and then the follow-on Fran from Microsoft Research. Forcing discretizations of continuous phenomena into an arbitrary serialization is just a way to kludge around a poor understanding of parallel activity.

So yeah, blackcoot shouldn't complain that X is broken, I'd say whatever app {s}he is writing needs to get fixed.
It isn't the app - it's the libraries the app is trying to use. They're a poor fit to the abstraction blackcoot would like to use.

Re:Multi-threading is GOOD [was Re:What I do] (3, Informative)

mike_sucks (55259) | more than 11 years ago | (#5100549)

First, you should restrict such broad generalizations to an X client in order to be even remotely correct.

Ah, I thought that was exactly what I implied. I was talking about the application, which is an X client. Of course the server needs to be thread and process safe - it does display multiple applications at once.

Wrong.

You're telling me multi-threaded apps and libraries are not harder to write, debug and maintain than single-threaded ones? Sorry: *you're* wrong. Even if you use a language which is designed with threading in mind (which X isn't), it adds a *lot* of complexity.

Also, having a single GUI thread is a good design pattern.
Um, add the hedge for some applications and I'll agree with you. Otherwise you're wedged in a one-track design mind.

Hey, I said it is a good design pattern. I didn't say it is a golden hammer. Of course you only apply a pattern when it fits.

You're making your problem fit your design, rather than your design fit your problem. There's any number of interactive applications that make lots of logical sense to be multi-threaded.

No, I'm not. Nor am I saying that applications should be single threaded. What gave you that idea?

There's no reason why an application needs to have a multi-threaded GUI. This is not to say that the application should not be multi-threaded, clearly that can very useful. The reason is because user interaction effectively serialized. A user rarely, if ever, provides multiple sources of input simultaneously. In cases when they do, it's usually supplementary to the interaction already occuring - a modifier. So there is rarely any need to process user input and update the display in multiple threads because there is only one thing going on at a time.

Go read papers on the "eXene" system in the programming language "ML". A pervasively multi-threaded X client library. One widget - one thread. It makes the widget code very easy to understand - you don't have to split your code into a bazillion little callbacks.

Damm, that sounds truely awful. What about user events that traverse mutliple widgets? How do you synchronise them all? What about the scheduling overhead when someone just drags a mouse over the app's UI and fifty threads are woken up almost simultaneously? Most workstations are still uniprocessor based.

And I don't see how that avoids the bazillion little callbacks issue. You're splitting your code up into bazillion little threads instead. In any case, the callbacks aren't an issue of you've designed and written your code properly.

Ever have to break up a callback into multiple functions, triggered by timers, just so the app didn't appear to "freeze" while you were off doing something time-intensive?

No, I just use a non-GUI thread to do the intensive work, so that the GUI thread is free to play with the user.

Multi-threading interaction can make the code much easier to maintain because you don't have to worry about "starving" parts of the application for events while busy working on others - the thread scheduler handles pre-emption for you.

Yeah, you "just" need to worry about synchronization, deadlocking, and other concurreny issues instead. Muuuuuuch easier. But what you said above made no sense to me (perhaps I need more coffee) - can you explain this in more detail?

Forcing discretizations of continuous phenomena into an arbitrary serialization is just a way to kludge around a poor understanding of parallel activity.

That would be the case if user interaction was a parallel activity, but unfortunately it is not.

It isn't the app - it's the libraries the app is trying to use. They're a poor fit to the abstraction blackcoot would like to use.

Ah, so you obviously know more about the app than I do, because I don't see any evidence in the article to support that statement.

/mike

Re:Multi-threading is GOOD [was Re:What I do] (2, Informative)

rpeppe (198035) | more than 11 years ago | (#5101980)


Yeah, you "just" need to worry about synchronization, deadlocking, and other concurreny issues instead. Muuuuuuch easier. But what you said above made no sense to me (perhaps I need some more coffee) - can you explain this in more detail?


It depends on the thread abstractions that are used for synchronisation and thread communication. The most commonly used abstractions today (semaphores, locks, etc) date from the 1970s; there are much better ways to do it!


One way derives from a mathematical notation created by Tony Hoare [ox.ac.uk] , called CSP [ukc.ac.uk] . There is one unit of thread communications and synchronisation, called a channel. It's like a rendezvous point that allows a value to be passed between threads. If one thread tries to send a value on a channel, it will block until another thread tries to read from the channel (also, reading from the channel will block until another thread tries to send on it).


This scheme is incredibly versatile, easy to use and cheap. There are also some tools [spinroot.com] that can aid in automatic verification of software built in this way. It's true that it's possible to deadlock in concurrent systems, but it's almost always possible to structure the system in such a way that it's deadlock-free by construction. For instance, if my program is structured as a one-way pipeline, it's impossible to deadlock.


Concurrency at this level in a GUI application can greatly enhance the simplicity and maintainability of a program. This is because it's generally much easier to write a straightforward piece of imperative code than encode the same thing as a state machine, e.g.



while (buttons != 0) {
(buttons, point) = <-mouse;
drawat(point)
}


(where <- receives from a channel), versus:


callback(buttons, point) {
if (state == DRAGGING) {
if (buttons != 0)
drawat(point);
else
state = NOTDRAGGING;
}
}


You say:
That would be the case if interaction was a parallel activity, but unfortunately it's not.


But it is! Yes, the user themselves only contributes one thread to the activity, but the program itself is often dealing with multiple activities at the same time; for instance updating itself in response to network activities or updating graphics on a time-step basis.


The most important thing it gives you, in my experience, is the sense of control. As a separate thread, you are free to structure your application in a way directly appropriate to the task being solved. In a callback system, you are at the mercy of the caller; you can't just wait for an event, then do the next thing, you have to encode your current state, return, and wait to be called back, whereupon you have to figure out where you just were!


For a language that exemplifies this, see Limbo [vitanuova.com] , the language of choice in the Inferno [vitanuova.com] environment. No problems with thread unsafe graphics there!

Re:Multi-threading is GOOD [was Re:What I do] (3, Informative)

Alex Belits (437) | more than 11 years ago | (#5103594)

And how does it help considering that the only usable general-purpose language is C?

Really, threads exist for one reason -- because OS developers write shitty schedulers, and because applications programmers don't understand their own data models and write shitty libraries. Most of things that "innovative" threads libraries do are various ways to implement serial communications between processes with asyncronous (or syncronous to asyncronous in your example) handling of events/messages. This is what pipes are for -- and yes, they work between threads, too. Just not in Windows.

Re:Multi-threading is GOOD [was Re:What I do] (1)

Pseudonym (62607) | more than 11 years ago | (#5115958)

And how does it help considering that the only usable general-purpose language is C?

Most new general-purpose code "out there" is written in C++, and with good reason.

Really, threads exist for one reason -- because OS developers write shitty schedulers, and because applications programmers don't understand their own data models and write shitty libraries.

You couldn't be more wrong. Threads exist because simpler solutions are not always appropriate.

If your code is effectively serialised, it's easier to write a serial program. If your code is well-partitioned, it's easier to fork off separate processes. It's those middle-ground cases for which you need threads. It's no accident that these also tend to be the hardest cases. As a result, some people think threads complicate programs, which they are not unless they are misused. Rather, threads are used to solve problems which are complicated to begin with.

Pipes are okay if that's what you need, but they're not appropriate for many IPC tasks. Priority-based message queues come close, but even with them, messages are packets of bytes, so you have to pay to marshal and unmarshal your data. (You also have to pay for the data to be copied twice since it is buffered inside the pipe/mq implementation, but this is an API problem. You really want your data copied straight between the address spaces without hitting a kernel or server buffer, as you would in a microkernel IPC primitive. I digress.)

Just handing over an abstract data structure or a pointer is far more efficient and convenient and, in some cases, the only tractable solution.

Re:Multi-threading is GOOD [was Re:What I do] (1)

mike_sucks (55259) | more than 11 years ago | (#5106131)

It depends on the thread abstractions that are used for synchronisation and thread communication. The most commonly used abstractions today (semaphores, locks, etc) date from the 1970s; there are much better ways to do it!

Well, the links you supplied do look interesting, and I've added them to my reading list, but the blocker for using more advanced thread syncing and comms is that they're not really usable in applications today.

When I'm designing software, I need to use technology that is available, proven and that the guys on my team understand, so unfortunately that counts most of the better solutions out (perhaps only for the time being). Also, you need to remember that we're talking about writing X client applications here, and so the better abstractions just aren't available anyway.

CSP seems interesting. It sounds kinda like the idea behind Java's PipedXXX streams and reader/writers. Have you got any references to other tools that support it? The two links that I found were both dead. :(

Concurrency at this level in a GUI application can greatly enhance the simplicity and maintainability of a program. This is because it's generally much easier to write a straightforward piece of imperative code than encode the same thing as a state machine, e.g.


[snip: a multi-threaded while loop and a single-threaded callback]

The callback *is* uglier, but your first example using the loop isn't terribly different from:

mouseDraggedCallback(buttons, point) {
if (buttons > 0) {
drawAt(point);
}
}

I'd say the suckiness of using a single thread (and hence of using callbacks) depends on the suckines of the toolkit you're using. If it has been designed properly, you'll get decent callbacks to use.

I'd also argue the callback above more clearly expresses the intention of the author than using a while loop. I think that is a huge boon to understandability and hence maintainability.

But it is! Yes, the user themselves only contributes one thread to the activity, but the program itself is often dealing with multiple activities at the same time; for instance updating itself in response to network activities or updating graphics on a time-step basis.

Okay, I was thinking more along the lines of user activity, rather than interaction. When the application is updating itself (on another thread) it isn't too hard a task to schedule a update on the GUI thread.

The most important thing it gives you, in my experience, is the sense of control.
[snip] In a callback system, you are at the mercy of the caller; you can't just wait for an event, then do the next thing, you have to encode your current state, return, and wait to be called back, whereupon you have to figure out where you just were!

I think that kind of situation is easily avoided by using seperate, non-gui, worker thread(s) to perform non-gui operations. You use the gui thread only for handling user interaction - both user events and updating the display.

The gui thread responds to user events by notifing the worker thread of anything significant. It acts as a translation layer between the user and the workers. For example, if a user selects File > Open, the gui thread prompts the user to select a file, then messages a worker thread to open the selected file. This is a powerful way of doing it because the worker thread can go off and load the file, and the gui stays responsive to further user input. It also keeps the application's code base clean, easy to understand and maintain because there is a clear separation between the gui code and the code that does the real work - the workers.

When the application needs to update the gui as a result of some change, the worker thread just queues a update event and the gui thread goes and performs the update. The gui thread obtains the state or data for the update from shared data sources. The gui thread reads the data, the workers update it.

You're still free to (mostly) structure your app how you want because the most important parts, the worker threads, are decoupled from the gui. You're also free to easily vary the gui (for example, use a different toolkit) without needing to change the whole app.

Re:Multi-threading is GOOD [was Re:What I do] (1)

JsTwO (97350) | more than 11 years ago | (#5107581)

what's the diff between this CSP thing and a message queue with explicit notify(win32 sendmessage)? u do understand how the system call write(...) works right?

to say the truth, there's only one true way to do the locking, all other ways r just same method in different expression.

Re:Multi-threading is GOOD [was Re:What I do] (1, Insightful)

Anonymous Coward | more than 11 years ago | (#5102081)

> You're telling me multi-threaded apps and libraries are not harder to write, debug and maintain than single-threaded ones? Sorry: *you're* wrong. Even if you use a language which is designed with threading in mind (which X isn't), it adds a *lot* of complexity.

Sure, when you're forced into a coding style out of the 1970's or earlier and have to write all the conditions, mutexes, semaphores, synchronizations, and so forth yourself.

I think the poster demonstrated quite plainly how much easier it is to write multithreaded code when the language and runtime actually supports it. I'll repeat what he said:

while (buttonisdown) followmouse()

Gets neater than that in languages that support generators. In haskell you filter out the events you want as a list comprehension over the list of events.

> There's no reason why an application needs to have a multi-threaded GUI.

There's no reason for you to have anything more than assembly code. If it makes you feel better, put a multithreaded GUI into one big synchronized block. The world is too full of people making excuses for obsolete design by saying it was meant to be that way forever.

Why add threads to xlib? (2, Insightful)

0x0d0a (568518) | more than 10 years ago | (#5098347)

Making xlib reentrant, and taking the associated performance hit, would be simply silly. Why should, say, snes9x, run slower because you want reentrant capabilities? A tiny percentage of apps have a multithreaded GUI -- you said that you were using Qt. Fine -- I could see support in Qt, but putting in Xlib would be a Bad Thing for the 99% of people out there that *don't* require multithreaded access to the GUI.

Secondly, threading a GUI program is a Bad Idea -- there's lots of interaction between threads in a GUI, and locking could turn into a nightmare. All you do in a GUI is usually check a value, paint something, and you have to lock all the values that might be shared.

Thirdly, it doesn't make sense for an event-based GUI. You have a thread that handles incoming events, maybe starts a *non-GUI* thread if you're really wedded to the idea of threading, and then gets the hell back to waiting for another event pronto so that you don't have a lot of latency on user input. Here's an example, an instance that might seem *on the surface* like a good place to do threading if X were reentrant. You have a GUI ftp program. One thread handles events, and each thread would download data and update the progress bar by repainting it. Bad idea. Now if I require a redraw, I have to sit around and wait for more data to come in. What I *really* want to have happen is have a counter for how far the progress bar is. When I require an update (periodically, and when a redraw is required), the GUI thread does it, checking the progress distance value. The non-GUI thread doing the transfer would set that value (in addition to a "dirty" flag) so that I get updates once a second or so, not 400 times a second if lots of little chunks of data are coming in. The GUI is simply a separate task from non-GUI tasks.

It has been for almost a decade (2, Informative)

acoopersmith (87160) | more than 11 years ago | (#5098842)

Xlib was made thread safe in the X11R6 [uni-bremen.de] release in 1994, but only if you initialize the locks it needs to do it properly via XInitThreads [xfree86.org] .

Re:It has been for almost a decade ... not quite (3, Interesting)

nathanh (1214) | more than 11 years ago | (#5099000)

Xlib was made thread safe in the X11R6 [uni-bremen.de] release in 1994, but only if you initialize the locks it needs to do it properly via XInitThreads [xfree86.org].

Xlib is thread safe but X11 is not. If your application has multiple threads then they must compete for the "display lock" with XLockDisplay and XUnlockDisplay. In other words, Xlib is used to solve the thread unsafety of X11. It's a library space (Xlib) solution to a protocol (X11) limitation.

So while you are right - Xlib is thread safe - you weren't answering the original question which is Why Isn't X11 Thread Safe? The answer is that it's not possible. He might as well ask why HTTP isn't thread safe, or why SMTP isn't thread safe. Imagine muxing two HTTP requests into a single request! It'd look like this:

GET /GET /index.htpage.asmlp HTTHTPTP/1./1.00

It's not an issue for the protocol to solve. This is why it's a non-problem. The questioner is simply confused. You don't make a protocol thread safe. You make the library calls to the protocol thread safe, and as you point out this has been solved with XFree86 for almost a decade.

Lots of Confusion! Long Explanation. (4, Informative)

nathanh (1214) | more than 11 years ago | (#5099482)

I'm going to explain myself more clearly because it's obvious from reading this thread that there is a LOT of confusion out there.

X11 is a protocol. Xlib is a C library that provides an API to the protocol. It is important to understand this distinction. Applications and toolkits do not have to use Xlib - they could generate X11 protocol streams directly, or they could use an Xlib replacement - but as there's nothing really wrong with Xlib nearly everybody uses it.

X11 is by design a client-server protocol. The client opens up a socket (UNIX socket, TCP/IP socket, etc) to the server. The client then sends multi-byte "messages" down the socket to tell the server to do stuff. For example, there is a message to draw a line. Each message has a few bytes to identify the command, then a bunch more bytes describing parameters to the command. The "line" message has one parameter describing which Window to draw to, one paramter for the Graphics Context (colour, line style, etc), and several parameters for the X,Y coordinates of the line.

Now imagine a threaded X11 client. Also imagine for the sake of argument that the client is generating X11 messages directly or is using a non-thread-safe Xlib. The client pseudo-code looks something like this:

Thread A:
while (1) { XDrawLine(); }

Thread B:
while (1) { XDrawArc(); }

Now remember that X11 is a protocol - a byte stream - so what is actually happening is that each thread is generating a sequence of bytes. The bytes look something like this:

Thread A Byte Stream:
LINE Display1 0 0 100 100 LINE Display1 0 0 100 100 ...

Thread B Byte Stream:
ARC Display1 0 0 50 360 ARC Display1 0 0 50 360 ...

Because these byte streams are both being fed down the same socket, and because the application is not thread safe, the resulting stream looks like this:

Combined Byte Stream:
LINE Display1 ARC 0 0 Display1 0 100 100 0 50 360 LINE Display1 0 0 LINE Display1 0 100 100 0 50 360 ...

It's an absolute mess! The X server gets very confused - it thinks the client has gone haywire - and so nothing works. There are only two solutions to this problem.

#1 is make all messages ATOMIC. This is simply impossible for sockets. You can make it work by getting rid of sockets and forcing all X11 clients to use a messaging IPC - and this IPC might even use sockets at the lowest layer - but it's impossible to retrofit it to sockets. The messaging approach has been used by Berlin, GDI, and a bunch of other windowing systems.

#2 is to force all multi-threaded X11 clients to impose their own locking. Each thread shares a lock for the protocol stream. Threads cannot proceed until they have gained the lock and for efficiency they should release the lock as quickly as possible. This is the approach that X11R5 (and X11R6) have used. Each thread uses XLockDisplay and XUnlockDisplay which are two new calls provided by Xlib. The change to the pseudo-code from before is trivial.

Thread A:
while (1) { XLockDisplay(); XDrawLine(); XUnlockDisplay(); }

Thread B:
while (1) { XLockDisplay(); XDrawArc(); XUnlockDisplay(); }

With this simple change in place your multi-threaded X11 client is now perfectly compatible with all X11 servers. The combined protocol stream is not confusing: the ARC and LINE messages are sequential rather than munged together.

Now the reason I think there is confusion here is that people are asking "Why Can't X11 be Multi Threaded?". The question is nonsensical. Socket protocols are not threadable. It's impossible to do this. It is very helpful here to understand that X11 is a LOT like other client-server protocols such as HTTP. In fact the analogies with HTTP are strong. HTTP has a client called the "web browser". HTTP has a server such as "Apache". The client opens a TCP/IP socket to the server. Messages begin with the multi-byte string GET /page HTTP/1.0. Optional bytes can follow describing additional HTTP functionality. The only real difference to HTTP is that X11 is PERSISTENT and has SERVER SIDE STATE. There are also minor differences such as the protocol is binary instead of text.

Now you can have a multi-threaded web browser, and you can also have a multi-threaded X11 client. You can have a multi-threaded web server, and you can also have a multi-threaded X11 server. But you can't have a multi-threaded HTTP protocol stream. Similarly you can't have a multi-threaded X11 protocol stream. It doesn't make any sense to even ask for this. As I showed before, it would be like a web browser requiring two URLs from a single server, but generating an HTTP "request" that looked like this.

GET GET /pag/index.htmle.asp HTTP/HTTP1.1.00

The solution is to serialise the HTTP commands in the web browser. The way to do this is with a serialisation library with locking. This is the same approach used by X11 with Xlib, provided by the XLockDisplay and XUnlockDisplay primitives.

You can reasonably argue that X11 wouldn't have this problem if it was a messaging protocol instead of a multi-byte stream protocol. That's the design decision that was made for X11, and I personally think it's a non-issue. There are other issues with the X11 protocol - it's quite heavy, many of the messages are limited or outdated, and some of the server-side state is useless - but the fact that is a BYTE STREAM protocol instead of a MESSAGING protocol is I think a non-argument. People seem to focus very heavily on it as the "reason that X11 sucks" but I think these people simply haven't investigated how the alternatives work. Eventually everything becomes a byte stream: it's just a design decision as to how early you make the conversion.

Even though (1)

CaptainZapp (182233) | more than 11 years ago | (#5101153)

I probably won't ever have an application for your explanations. Neverless it's posts like this, which (still) make this a very interesting forum.

Thanks for the education.

Re:Lots of Confusion! Long Explanation. (1)

X (1235) | more than 11 years ago | (#5105412)

In general, this is a very informative post. I'd like to point out a couple of things:

1) The IMAP protocol, HTTP protocol, and for that matter many other protocols support the notion of simultaneous processing of multiple requests. You may not be able to submit requests exactly simultaneously, but you don't have to wait for the response from one before you submit the next request. The same is true of X11. So, the only "atomic" part of X11 sending and parsing of X messages, and this is only if they are sharing the same connection.

2) There is nothing which prevents a multi-threaded application from establishing multiple connections with an X server. This would allow for completely multithreaded operations (well, maybe the graphics card driver would impose some synchronization points, but who knows).

Re:Lots of Confusion! Long Explanation. (0)

Anonymous Coward | more than 11 years ago | (#5110354)

Nathan gives a good explanation of how the basic locking stuff works, and why. I can give some history (although, this being /., I doubt anyone will actually read a post to a 2-day old thread).

The simple lock/unlock mechanism was added as an almost theoretical exercise to X11R6, but we at the X Consortium really didn't know how to test it and the other serializations in Xt, Xm, and Xdt. I know that, because it was my job to test them. Only Sun really took it up and produced a test suite for X11. The reference source was and I'm guessing remains flaky and inefficient.

Moving forward, at Microsoft (my current employer) we had the job of porting Internet Explorer, which makes pretty heavy use of multithreaded GUI, to UNIXes. We couldn't rewrite much, and we used the Mainwin emulation layer, and by the way daily sync with the Windows IE sources. Working very closely with Mainsoft, we figured it all out on Solaris and there are no problems associated with its multithreading nature (ignoring the instinctive funny comments I know this will draw, but it's true). Unfortunately, Sun kept accidentally breaking MT in Xlib patches, something that was pretty easy to show with a simple test app, but of course nobody would believe this and we got the blame for frequent startup hangs. Eventually Sun sorted things out (it took about 9 months) but this explains why we have been paranoid about patch levels in later IE/UNIX releases. It did lead me to the ironic conclusion that Microsoft got into trouble because we, of all companies, released an app that was operating at he bleeding edge of X11 and UNIX -- I suspected that IE was the only widely used client that had multithreaded GUI.

When time came to port to HP-UX, we had to use the reference X11R6 source, because HP-UX 11 was still not out, and 10.1 used R5, We got persistent hangs. One of our devs found a race condition by reading the code and looking for the standard incorrect patterns (it was an incorrectly handled "do" loop, as I recall, but that's all the detail I remember). Because we never saw the same problem in Solaris, we assumed they had quietly fixed it in their libraries. I told the Open Group about the fix, but they didn't seem interested. For all I know, the race is still there.

Addressing Markus Registrada, there are static areas in Xlib and Xt that need MT protection, so it's not safe to open multiple threads even if each has private access to display resources.

No longer coding in X or UNIX:

-- David Brooks

Re:It has been for almost a decade ... not quite (0)

Anonymous Coward | more than 11 years ago | (#5100016)

You are a pedant or an idiot or both.

Re:It has been for almost a decade ... not quite (0)

Anonymous Coward | more than 11 years ago | (#5100305)

You are ignorant and a dickhead which is why you have no friends.

Qt is thread safe (2, Interesting)

frezeal (133571) | more than 11 years ago | (#5099008)

If you link against libqt-mt, you can post events outside of the main event thread. You still shouldn't call methods directly on QtWidgets. But the link should answer all of your questions and solve your problem.

http://doc.trolltech.com/3.0/threads.html

a little clarification... (3, Informative)

blackcoot (124938) | more than 11 years ago | (#5099915)

i'm working on a real time computer vision system. capture runs in its own thread, firing off imageArrived events which end up being executed in the capture thread (a subtlety of qt that i was unaware of). this imageArrived event gets plugged into whichever listeners are interested, the idea being to allow multiple paths of processing the same image. i attempted to do this using timers, having capture events posted peridiodically and then occasionally refreshing. this was very unsatisfactory for two reasons: 1) it looks awefully slow even though i know that it's running very quickly underneath and 2) (more importantly) this method will only ever use a single processor. as i expand to do more interesting processing, i'm going to be forced to use multithreading because a single processor will not be able to do it all in the allotted amount of time, however, multiple processors could because of how much of these processes are easily parallelizable. for a rough guestimate of how much processing is involved, there are four classifiers per pixel (three based on chrominance or one based on luminance and two based chrominance plus a classifier that combines the output of the three other classifiers), 640x480 pixels 30 frames per second for a grand total of 37 odd million classifications per second. add to this the possibility of yuv to rgb conversion, and contraction/expansion filtering to clean up noise and you can quickly see how the time adds up. this is before i even begin to do anything useful with the images --- this is just removing the boring pixels. so far, this is all integer arithmetic on arrays that are "cache friendly" in their layout. the really cpu intensive stuff comes later (this is all preprocessing in terms of my application)... updating classifier's background models, extending classification to foreground, background, and things that have been merged into the background (i.e. have stayed still long enough, either items removed from the scene, added to the scene or moved within the scene).

for those who were wondering, i did figure it out with a number of critical sections synchronized indirectly on xlib's mutex. performance is actually better threaded than with earlier single threaded prototypes (mostly because i am now able to start processing an image while i start getting the next one). looking back, i realize that most of my frustration is the result of the vast majority of my gui programming experience being done on windows in applications that were fundamentally stupid to thread. as the message that cliff pointed out notes, this particular quirk of x is not very well documented and has the potential to be very counter intuitive to people doing this for the first time.

anyways, thanks a lot for the help. for those who are curious, my goal is to release the source for the framework and sample application within a month.

Don't Bother (2, Interesting)

Markus Registrada (642224) | more than 11 years ago | (#5099919)

There's no reason for the X wire protocol to be "thread-safe" (i.e. atomic/transactional). There's not even any need to use locks in Xlib.

A multithreaded program may simply open several connections to the X server, and operate independent UIs. One thread might operate a display window, while another operates a dialog box or progress bar. As far as X is concerned, it's talking to different programs.

Don't go looking for complicated solutions when simple ones are right at hand.

Limited connections enforced by OS? (1)

yerricde (125198) | more than 11 years ago | (#5113836)

A multithreaded program may simply open several connections to the X server

Unless your X server severely limits the number of connections available to it, to force you to pay for X advanced server or X datacenter server. I've been told that programs such as X servers running on Microsoft Windows operating systems are subject to a limitation on the total number of incoming TCP connections.

Open a new display (1)

dfgdfgdfg (577386) | more than 11 years ago | (#5100791)

are there fundamental design decisions in X11 that make dealing with receiving events from multiple threads simultaneously, impossible?

You can receive events from different threads. In the other threads, open a new display using XOpenDisplay(). You can then use use XSendEvent() to send an event to the window you want. Window IDs are share between all X clients.

:p (0)

Anonymous Coward | more than 11 years ago | (#5102032)

haha windows has been thread-safe since NT 4
a big :P to all you linux programmers.

Uh, I dunno... (1)

iankerickson (116267) | more than 11 years ago | (#5103948)

Ok, I give up. Why Isn't X11 Thread-Safe?

Nobody spoil the punchline. I haven't heard this one yet.

Re:Uh, I dunno... (1)

Royster (16042) | more than 11 years ago | (#5104687)

Becuase Bill Gates.

Get it? Because Bill *Gates*. Ho ho ha ha ha ha... ha... heh heh heh ... heh...

<whew!> /me stops to catch his breath.

Re:Uh, I dunno... (1)

iankerickson (116267) | more than 11 years ago | (#5111243)

Thanks. I'd hoped that given enough monkeys and enough typewriters that Slashdot wouldn't let me down...

Why Isn't X11 Thread-Safe? (0)

Anonymous Coward | more than 11 years ago | (#5109757)

Because it's a bloated piece of shit that should have been put out of its fucking misery years ago?

Just wonderin'.
Check for New Comments
Slashdot Login

Need an Account?

Forgot your password?

Submission Text Formatting Tips

We support a small subset of HTML, namely these tags:

  • b
  • i
  • p
  • br
  • a
  • ol
  • ul
  • li
  • dl
  • dt
  • dd
  • em
  • strong
  • tt
  • blockquote
  • div
  • quote
  • ecode

"ecode" can be used for code snippets, for example:

<ecode>    while(1) { do_something(); } </ecode>