Beta

Slashdot: News for Nerds

×

Welcome to the Slashdot Beta site -- learn more here. Use the link in the footer or click here to return to the Classic version of Slashdot.

Thank you!

Before you choose to head back to the Classic look of the site, we'd appreciate it if you share your thoughts on the Beta; your feedback is what drives our ongoing development.

Beta is different and we value you taking the time to try it out. Please take a look at the changes we've made in Beta and  learn more about it. Thanks for reading, and for making the site better!

Comments

top

Ask Slashdot: Where Do You Get (or Share) News About Open Source Projects?

Mr Thinly Sliced Re:Slashdot (85 comments)

"Phoronix Forums. You will never find a more wretched hive of scum and villainy. We must be cautious."
- Obi-Wan Kenobi

2 days ago
top

Linux Needs Resource Management For Complex Workloads

Mr Thinly Sliced Re:complex application example (161 comments)

You're welcome - I hope you get it sorted out.

The only other thing I'd mention - you perhaps noticed I kept saying "threads like.." and "with regular threads" because it's basically introduced a number of single points of failure. Due to the lack of back channel or retransmission, things can go silently wrong without notice (network cable failure etc). In an ideal world you'd double up on some of that infrastructure and networking.

I know you need to get something up and running, but it's perhaps something to bear in mind for a later iteration.

about a week ago
top

Linux Needs Resource Management For Complex Workloads

Mr Thinly Sliced Re:complex application example (161 comments)

FWIW I agree vis-a-vis using UDP for a business critical thing. I'd want exemption from responsiblity for any missed packets purely due to the infrastructure in between.

about two weeks ago
top

Linux Needs Resource Management For Complex Workloads

Mr Thinly Sliced Re:complex application example (161 comments)

First - the problem with python is that because it's a VM you've got a whole lot of baggage in that process out of your control (mutexes, mallocs, stalls for housekeeping).

Basically you've got a strict timing guarantee dictated by the fact that you have incoming UDP packets you can't afford to drop.

As such, you need a process sat on that incoming socket that doesn't block and can't be interrupted.

The way you do that is to use a realtime kernel and dedicate a CPU using process affinity to a realtime receiver thread. Make sure that the only IRQ interrupt mapped to that CPU is the dedicated network card. (Note: I say realtime receiver thread, but in fact it's just a high priority callback down stack from the IRQ interrupt).

This realtime receiver thread should be a "complete" realtime thread - no malloc, no mutexes. Passing messages out of these realtime threads should be done via non-blocking ring buffers to high (regular) priority threads who are in charge of posting to something like zeromq.

Depending on your deadlines, you can make it fully non-blocking but you'll need to dedicate a CPU to spin lock checking that ring buffer for new messages. Second option is that you calculate your upper bound on ring buffer fill and poll it every now and then. You can use semaphores to signal between the threads but you'll need to make that other thread realtime too to avoid a possible priority inversion situation.

> how do you then ensure that the process receiving the incoming UDP messages is high enough priority to make sure that the packets are definitely, definitely received

As mentioned, dedicate a CPU mask everything else off from it and make the IRQ point to it.

> what support from the linux kernel is there to ensure that this happens

With a realtime thread the only other thing that could interrupt it would be another realtime priority thread - but you should make sure that situation doesn't occur.

> is there a system call which makes sure that data received on a UDP socket *guarantees* that the process receiving it is woken up as an absolute priority over and above all else

Yes, IRQ mapping to the dedicated CPU with a realtime receiver thread.

> the message queue destination has to have locking otherwise it will be corrupted. what happens if the message queue that you wish to send the UDP packet to is locked by a *lower* priority process

You might get away with having the realtime receiver thread do the zeromq message push (for example) but the "real" way to do this would be lock-free ring buffers and another thread being the consumer of that.

> what support in the linux kernel is there to get the lower priority process to have its priority temporarily increased until it lets go of the message queue on which the higher-priority task is critically dependent

You want to avoid this. Use lockfree structures for correctness - or you may discover that having the realtime receiver thread do the post is "good enough" for your message volumes.

> to the best of my knowledge the linux kernel has absolutely no support for these kinds of very important re-prioritisation requirements

No offense, but Linux has support for this kind of scenario, you're just a little confused about how you go about it. Priority inversion means you don't want to do it this way on _any_ operating system, not just Linux.

about two weeks ago
top

Linux Needs Resource Management For Complex Workloads

Mr Thinly Sliced Re:complex application example (161 comments)

> the first ones used threads, semaphores through python's multiprocessing.Pipe implementation.

I stopped reading when I came across this.

Honestly - why are people trying to do things that need guarantees with python?

The fact you have strict timing guarantees means you should be using a realtime kernel and realtime threads with a dedicated network card and dedicated processes on IRQs for that card.

Take the incoming messages from UDP and post them on a message bus should be step one so that you don't lose them.

about two weeks ago
top

Ask Slashdot: An Open Source PC Music Studio?

Mr Thinly Sliced Re:Never gonna happen, because of how OSS works. (299 comments)

> Why on earth would you want to mix your professionally sounding audio with every freaking app that goes "ding" or "boing" or "you've got mail"?

Because at it currently stands any application that attempts to output through pulseaudio will get blocked and possibly hang until you stop jackd.

Having to "stop" jackd and my audio mixing session just because I want to take a skype call is not acceptable in 2014.

about 6 months ago
top

Intel Dev: GTK's Biggest Problem, and What Qt Does Better

Mr Thinly Sliced Re:GTK is trash (282 comments)

I don't know if it still is - but IIRC VMWare's Linux client was GTK based.

I haven't seen any others though tbh - but I don't use a lot of commercial software on Linux.

about 6 months ago
top

X.Org Server 1.15 Brings DRI3, Lacks XWayland Support

Mr Thinly Sliced Re:How well does XWayland work? (340 comments)

Yes. I've hacked aroud(sic) in xlib a lot.

Splendid - then I won't need to copy-paste some of the problems regarding right click menus locking the server and similar issues documented elsewhere - right? The X developers that have moved over to developing Wayland have explained these issues far better than I can.

Would you be kind enough to acknowledge these problems exist?

I get the feeling that people have invested so much energy and effort in X that they're reluctant to acknowledge the problems even exist. It's like some kind of programming Stockholm Syndrome.

I started recording live TV on a Linux box in 2003. I never had problems. What were you trying to make?

A simple application capturing part of the desktop plus a live video feed with V4L and synchronised audio capture.

I started with just trying to capture the desktop + audio. Using either direct ALSA, pulseaudio or jack I would get scheduling issues and dropped X11 frames and/or audio underflow sometimes skipping multiple screen capture frames at once. Once I saw the problems programmatically I tried using the raw ffmpeg binary which exhibited the exact same issues either under a regular or realtime kernel. I did try with a recent compiled version, too.

We are getting way off topic with this discussion however. I feel my original premise (OSX is not behind X in terms of functionality or libraries) still stands.

about 7 months ago
top

X.Org Server 1.15 Brings DRI3, Lacks XWayland Support

Mr Thinly Sliced Re:How well does XWayland work? (340 comments)

> You know none of that stuff is in wayland either?

Yep, I was replying to GPs assertion that OSX has to catch up to X11.

> Also what's wrong with X event handling? You can select() on the events just like any others.

Have you ever written a real X protocol application or used one? Xedit, for example. The code is a mess of anachronisms (events targeted to sub-windows - if you are using them - modern toolkits don't now, excessively verbose error handling due to the historical cruft in the protocol etc) that haven't been brought into the 20th century due to the backwards compatibility needed.

Now you can say "yes, but these days everyone uses toolkit ##" - and you'd be correct, and you'd be using the toolkits abstraction on top of the X protocol and events. It gets messier when you want to add a sub-window with a GL or YUV surface. Now you're dealing with X, Toolkit ##, XGL and GL.

> Also, I've touched on OSX video handling. This is not a high point of OSX. Decoding a video and getting at the pixels is far, far better under ffmpeg than OSX.

Back when I moved onto OSX it took me about three days to get a custom screen + audio capture program up and running (with live preview) - something I was unable to getting working under X/ffmpeg and ALSA/pulseaudio without frame drops and synchronisation problems after many days of going grey. Feel free to blame it on my programming skill - but I found Apple's documented examples far easier to use as a base. Even using the ffmpeg binary alone with X11 capture and alsa/jack had frame drops and stalls in capture.

Perhaps this is a case of "better the library you know"? For me the OSX video handling is great and simple to use. Were you using the Objective-C bindings or talking C?

about 7 months ago
top

X.Org Server 1.15 Brings DRI3, Lacks XWayland Support

Mr Thinly Sliced Re:In a country far away and long ago.... (340 comments)

Friends, Country Men (and Women)!

This propaganda is exactly that which the Royalists would have you believe - without telling the complete story of waste and unsightliness they would foist upon us once again should we believe their lies!

Our royalist friend has failed to mention that under the Kings rule, some houses would remain half painted for a full day! It was due to the King having only limited time with which to grant delegation powers to the Minister of Composition who was powerless to get painting done without the Kings intervention.

Perhaps our royalist friend would also like to explain the excessive bureaucracy surrounding simple requests like even paint colors? Every simple request had to pass the Kings desk.

I for one would also like an explanation for the massive expense of maintaining a whole legion of Royally Approved color choosers that could only choose baroque antique color schemes that nobody wanted or commissioned.

No my Friends, the Royalists have but one agenda - to return us to the times of half painted houses and top heavy bureaucracy that punctuated our time as a Monarchy.

about 7 months ago
top

X.Org Server 1.15 Brings DRI3, Lacks XWayland Support

Mr Thinly Sliced Re:How well does XWayland work? (340 comments)

> Having seen terrible X compatibility layers for Mac OS X and Windows

The OSX X (XQuartz) implementation _is_ xorg-server (currently 1.14.4) - you know, the one used on Linux (with certain OSX specific tweaks to allow non-root mode)

The problems you mention with interoperability are largely down to the core windowing systems being vastly different models. We can argue about which model is correct but the interoperability problems are a side effect of different models - not evidence of a particular model being bad.

I'm not convinced from your descriptions here you quite understand the complexity of the interactions.

> The lack of X has been the main technical drawback Mac OS X has been
> having compared to Linux.
> I'd much rather see Mac OS X catch up with Linux than for Linux to go down
> to the level of Mac OS X.

OSX has vsync based updates, sensible event handling and lots of core library stuff (like the AVFramework) that makes it a pleasure to program compared to XWindows.

The Linux desktop _needs_ to get off X. It's an outdated behemoth with a model that is way out of date. Now you could say "well let's update the model then".

Sure, you can do that. And when you do that, you get Wayland.

about 7 months ago
top

YouTube Adds Play Icon To Page Titles To Show Which Tabs Are Making Noise

Mr Thinly Sliced Re:Belong in the browser, maybe? (150 comments)

Re:Belong in the browser, maybe?

If the Redhat guys have taught me anything it's that it belongs in systemd.

about a year ago
top

Using Java In Low Latency Environments

Mr Thinly Sliced Re:C++ can operate at the very limits (371 comments)

Pfft. I prototype in Java 7, then implement the real thing in C++.

I notice you haven't actually refuted any of my points, by the way. Go away, troll.

about a year ago
top

Using Java In Low Latency Environments

Mr Thinly Sliced Re:C++ can operate at the very limits (371 comments)

There is no reason Java can not run just as fast and predictably as C/C++, given people who know what they are doing.

Yes, yes there is. Well, more than one actually. Here we go again:

(1) Unbounded stop the world of the JVM causing thread stalls (GC being the canonical example)
(2) Other stop the world's in the JVM due to code profiling causing a newly compiled version to be swapped in (maybe they've switched to atomics here, I'm not sure)
(3) Java's internal profiling and housekeeping absorbs CPU cycles
(4) Everything is allocated on the heap. If you have an array of objects - it's an array of pointers to things in the heap. There is no packed array of structs

These are the ones I've run up against when attempting to reduce scheduling latency in Java.

You can reduce the impact of (1), (2) and (3) using a realtime Java VM - but then you're not really writing Java - you can't use standard containers or standard libraries as they allocate memory all over the place. Also, the ahead of time compilation means you've lost the benefit of that lovely JIT re-profiling and optimisation.

(4) Can be mitigated a little by using NIO buffers with long offsets - but it's a mess and adds unnecessary extra computations to something that should just be pointer + offset.

To get down to it - with the standard Java VM I get scheduling latencies occasionally larger than 20ms - with C++ my thread scheduling latencies are measure in nanoseconds (i.e. less than a microsecond).

This is on a real time kernel with all appropriate thread priorities and such.

about a year ago
top

Using Java In Low Latency Environments

Mr Thinly Sliced Re:performance is not black and white (371 comments)

Being a big "C" booster these days, most of my hate is reserved for C++. It is a language with no home. Overly complicated, not as lightweight as C, and not as easy to use as Java or .NET. I could speak at length about its problems.

I'll be the first to admit to the myriad of problems with C++ (compilation speed and difficulty refactoring being my pet peeves) - but it has things that just make life more pleasant over C:

* RAII
* Standard library for algorithms, structures and so on
* Abstraction using templates are free - (well, one byte on the stack) and let you do (admittedly sometimes crazy) flexible things
* Abstraction at all
* Exceptions

You're welcome to your opinion, of course. I feel that C is overly simplistic and encourages re-implementation of the most mundane of things.

One look at things like GObject and GTK should at the very least make you question the sanity of accounting for every little thing. C doesn't leave you any choice - you must micromanage everything.

about a year ago
top

Using Java In Low Latency Environments

Mr Thinly Sliced Re:Warm up is a big deal (371 comments)

The standard fix seems to be to write apps to exercise their own critical paths at startup. This is *hard* when dealing with front-end code on the edge of the system you control. Even when it's easy, it's still something you have to do in Java that is entirely irrelevant in compiled languages.

These people should be using a real time Java VM and an ahead of time compiler for those parts.

It means you can skip the warm up. The only problem with it is you can't hire regular Java guys - real time Java is a little bit special, and you can't use regular Java collections and other things in the ways you'd expect.

about a year ago
top

A Radical Plan For Saving Microsoft's Surface RT

Mr Thinly Sliced Re:Support costs (391 comments)

Look up DG Trade and Undertakings.

MS would be on very shady ground indeed in the EU if they tried it here.

1 year,5 days
top

Next-Next Generation Video: Introducing Daala

Mr Thinly Sliced Re:Essential Features.. alpha support, lossless. (86 comments)

Have you tried contacting the Xiph guys at all?

If email isn't working try IRC - they probably have a channel the devs hang out in, or their mailing list.

You have a nice suggestion and good grounds for it being a feature - do a little lobbying and see if you can get one of the devs to champion the feature for you!

about a year ago
top

Oracle Discontinues Free Java Time Zone Updates

Mr Thinly Sliced Re:Java at least as fast as C++ (405 comments)

However, modifying the C++ version to utilize custom new and delete operators for the necessary classes to cater to the specific demands of such an application once again pushes the C++ version's performance ahead of Java, as expected.

I'm aware of such tests - but all they confirm is that pre-allocating a chunk of memory and re-using it is faster than individual allocate/deallocates.

That's not really that surprising. vector and other containers in C++ provide methods for storage (pre)allocation that can perform similarly.

If anything, it indicates that the program code is probably less than stellar C++ (stack allocations won't cost you anything and your destructors should be empty for value types).

Modern C++ shys away from OO derivations, too, where you should be using templates to get your abstraction for (almost) no runtime cost. Don't use virtual function calls - they're expensive - derive from a template class or use functors and get an inlined call for (almost - one byte) free.

Now don't forget Java has other things working against it (class recompilation, profiling, JVM stalls due to housekeeping etc).

A native compiled C++ program won't stall or halt your running threads. It's predictable.

Now we can argue about the effort to write decent Java and decent C++ and there I'll agree Java is a much more approachable language.

about a year ago

Submissions

Mr Thinly Sliced hasn't submitted any stories.

Journals

top

Early Career Laughs

Mr Thinly Sliced Mr Thinly Sliced writes  |  more than 7 years ago

Copied from a reply I made to Luke727 which brought back some memories:

Back early in my career, I worked for the first ISP in the UK (UKOnline), and myself and a good friend were charged with coming up with the 'first password' routine for the service.

Now Joe being Joe, decided the best way would be to use a dictionary, and then provide, if you will, an anti-dictionary - a dictionary full of swear words.

We would take two words based on this algorithm, and separate them with a dot.

Joe sent an email around the offices asking everyone to send back an email with all the swear words they knew, so that we could use it as the anti-dictionary, and filter the real dictionary based on it.

You wouldn't believe some of the mails that Joe received - some really dirty ones from people you wouldn't expect, either.

Anyways, the system was setup and sure enough, if a word was found in the dirty dictionary, it wasn't used as one of the pieces of the password.

Sure enough, three weeks after going live, customer support received a really annoyed and offended call from a female customer who had received:

moist.panties

I shit you not.

Slashdot Account

Need an Account?

Forgot your password?

Don't worry, we never post anything without your permission.

Submission Text Formatting Tips

We support a small subset of HTML, namely these tags:

  • b
  • i
  • p
  • br
  • a
  • ol
  • ul
  • li
  • dl
  • dt
  • dd
  • em
  • strong
  • tt
  • blockquote
  • div
  • quote
  • ecode

"ecode" can be used for code snippets, for example:

<ecode>    while(1) { do_something(); } </ecode>
Create a Slashdot Account

Loading...