Welcome to the Slashdot Beta site -- learn more here. Use the link in the footer or click here to return to the Classic version of Slashdot.

Thank you!

Before you choose to head back to the Classic look of the site, we'd appreciate it if you share your thoughts on the Beta; your feedback is what drives our ongoing development.

Beta is different and we value you taking the time to try it out. Please take a look at the changes we've made in Beta and  learn more about it. Thanks for reading, and for making the site better!

Designing a Programming Language For Embeddability

timothy posted more than 3 years ago | from the get-in-there-you dept.

Programming 52

CowboyRobot writes "The creators of the Lua language describe the process of designing a new language and the constraints that certain parameters, specifically embeddability, place on the process. 'Many languages (not necessarily scripting languages) support extending through an FFI (foreign function interface). An FFI is not enough to allow a function in the system language to do all that a function in the script can do. Nevertheless, in practice FFI covers most common needs for extending, such as access to external libraries and system calls. Embedding, on the other hand, is harder to support, because it usually demands closer integration between the host program and the script, and an FFI alone does not suffice.'"

Sorry! There are no comments related to the filter you selected.

<body><head uuid="0"></body> (0)

Anonymous Coward | more than 3 years ago | (#36172696)

<head uuid="0"/>
<body head="0">
'nuff said

Oh good (0)

MightyMartian (840721) | more than 3 years ago | (#36172702)

Oh good, another language.

Re:Oh good (4, Informative)

pem (1013437) | more than 3 years ago | (#36172762)

It's 18 years old. If you don't care to know anything about existing languages, why would having newer ones bother you?

Re:Oh good (2)

rrohbeck (944847) | more than 3 years ago | (#36173036)

Bah. Newfangled crap. I'll stick to COBOL, thank you very much.

Re:Oh good (0)

Anonymous Coward | more than 3 years ago | (#36172772)

Your ignorance is showing. Here, I'll give you a geek card to cover it up.

If they are going to, might as well do it right. (3, Interesting)

pavon (30274) | more than 3 years ago | (#36172790)

Since it's a given that people are going to be creating new languages, I certainly hope they listen to what Roberto has to say. Lua is a joy to embed into applications compared to python and others.

Re:If they are going to, might as well do it right (-1)

Anonymous Coward | more than 3 years ago | (#36172842)

Can you compare it to a language which doesn't suck cowboy neil's sweaty asshole?

Re:If they are going to, might as well do it right (0)

Anonymous Coward | more than 3 years ago | (#36186308)

Lua is a good embeddabilibuddy.

Worst (0)

TheDarkMaster (1292526) | more than 3 years ago | (#36177130)

Another interpreted, garbage-collected, slow as hell but easy to use than evil C language... When this madness will end?

Don't overlook the latest in maths discoveries. (0)

OpenQL (1754612) | more than 3 years ago | (#36172954)

That reminds me i really should start the OpenQL programming language, that exists not for reasons of grammar but the perfection of maths our true evolutionary route. Anyone interested in assisting the project mail me openql ..a..t gmail. Peace.

Re:Don't overlook the latest in maths discoveries. (0)

Anonymous Coward | more than 3 years ago | (#36173374)

It already exists. And it's called Haskell. ;)

Lua's Intermediate Representation (0)

Anonymous Coward | more than 3 years ago | (#36172982)

Interestingly, Lua does not offer a stable/standardized IR form (i.e. they offer no guarantees that it'll not change from version to version). Expressing a program in IR form is valuable in that you can apply a lot of existing IR optimizations to it to gain performance (IR is a representation somewhere in between abstract syntax trees and low level machine code symbolic representation.), e.g. dead code elimination, constant propagation, copy propagation, strength reduction, common subexpression elimination, loop unrolling, loop fission/fusion, memory layout optimization, structure splitting, array reshaping, etc.

Some of the optimizations I mentioned are key in V8 and WebMonkey's performance gains...however you do get the usual caveat. Such analysis takes time and if you're running a run-once and throw-away code snippet, you have to think if it's worth the trouble.

I think Lua can afford to do this, since most hosts that use Lua runs Lua scripts for a long period of time (e.g. a Game that runs a Lua script for controlling an AI). Applying some of the optimizations mentioned earlier should boost some of these codes' performance by a lot.

Re:Lua's Intermediate Representation (-1)

Anonymous Coward | more than 3 years ago | (#36173062)

pull your finger out of your asshole, chump.

Re:Lua's Intermediate Representation (1)

Anonymous Coward | more than 3 years ago | (#36173532)

I think the main reason Lua doesn't have a stable IR form is because in practice there simply isn't a need for the kinds of optimizations it would enable. Lua firmly establishes itself as an embedded scripting language, so in nearly all practical cases whatever Lua code is included in your application represents a negligible amount of resources used (or at the very least isn't one of the big bottlenecks). This is on top of the fact that Lua code already is very fast for a scripting language. If Lua performance is really needed, a JIT compiler exists that'd improve performance to the point of making further optimizations more or less irrelevant.

V8 and WebMonkey, conversely, need such optimizations because the JavaScript code becomes the bottleneck in big web applications. In their case, the interpreter needs to be as fast as possible, which makes every optimization useful.

Forth (3, Informative)

smoothnorman (1670542) | more than 3 years ago | (#36172990)

Forth [] was designed to run on (albeit early) embedded platforms. Extensible, easy to learn, easy to implement. I guess it's just the RPN that scares folks off? ("no you old fool! modern processors are optimized away from stack intensive use!")

Re:Forth (1)

nielsm (1616577) | more than 3 years ago | (#36173410)

Except this is not about languages running on embedded platforms, this is about hosting a language inside an application, for providing e.g. scriptability of a GUI application.

Re:Forth (2)

VortexCortex (1117377) | more than 3 years ago | (#36173432)

Wrong kind of "embedded" -- The Lua devs meant "easily accessed from within the host language ( C / C++ )".

Re:Forth (1)

Darinbob (1142669) | more than 3 years ago | (#36173782)

I always liked REXX myself, and AREXX on Amiga. Simple, easy to embed, the host program is in control as opposed to the embedded language's loop, integrates with other programs using the same embedded language, etc.

Re:Forth (0)

Anonymous Coward | more than 3 years ago | (#36178910)

"Atlast" is an example of a forth that is intended for that kind of embedding. It's kind of showing its age, but it seems like it would be practical.

It's over at fourmilab [] , and is public domain.

Re:Forth (1)

hoggoth (414195) | more than 3 years ago | (#36173492)

Loeliger's book 'Threaded Interpreted Languages' was one of my favorites non-fiction books ever. (Yes, I am that geeky).
Reading how he started with just a few tens of opcodes and chapter by chapter boot-strapped an entire operating environment was like reading Genesis.

Re:Forth (1)

smoothnorman (1670542) | more than 3 years ago | (#36173524)

yeah! it along with the later one by Brodie always comes to mind when i'm writing embedded applications (in either of the disparate senses of "embedded". should be a clear way to specify them .. machine-embedded or Gui-bedded ?) []

Apparently Lua needs some press (0)

Anonymous Coward | more than 3 years ago | (#36173020)

All of the people I know who have experienced Lua (5 or so) think it's inferior. One even works at a company that embedded Lua into their product as a scripting language only later to realise that was big mistake.

Re:Apparently Lua needs some press (1)

Musc (10581) | more than 3 years ago | (#36173120)

Inferior to what?

Re:Apparently Lua needs some press (-1, Troll)

larry bagina (561269) | more than 3 years ago | (#36173734)

embedding a hard dick in a wet vagina.

Re:Apparently Lua needs some press (1)

lahvak (69490) | more than 3 years ago | (#36174082)

It would be helpful if you included some details.

Re:Apparently Lua needs some press (1)

LordLimecat (1103839) | more than 3 years ago | (#36174584)

World of Warcraft uses LUA for mods, and noone seems to be having any problems with it. If anything, theres too many mods.

Re:Apparently Lua needs some press (1)

tibit (1762298) | more than 3 years ago | (#36178052)

Hilscher uses it in some of their communications gateways. From my personal experience in one particular application, the interpreter was too slow to move all the bits and bytes that had to be moved between two systems speaking different protocols, while keeping the communications (machine) cycle at 1ms. YMMV of course, so this is just a specific case, not a generalization.

I think it'd have been much easier for me as a customer if they simply provided an export library exposing relevant APIs in their firmware, provided remote gdb interface, and provided customers with the build of the open source C compiler that they most likely use themselves (their chips use an ARM core). You'd then upload the linked "executable" into the system, it'd get linked there, and executed just like Lua interpeter is being executed.

I don't see any direct benefits from using scripting languages where it's not hard at all to link your own code into the running system. In many embedded applications, you need realtime and deterministic performance. With any garbage-collected language it means that every machine/process cycle when you need to do garbage collection, to keep the overhead distributed across machine cycles. If you do GC too rarely, it may take too long and won't fit into a machine cycle any longer.

Scripting languages kinda lose their appeal if the API exposed to the scripting language has 1:1 mapping to the underlying C API. High-level languages like Python or Lua are supposed to provide benefits because you program at a higher level of abstraction than bare C. Yet if the interface to the system is very low level, you end up writing C-like code in the scripting language. That's especially true if your functionality is fairly simple -- there's no reason to introduce additional layers of abstraction in your own code, after all. Yet the underlying abstractions in the platform you work on may have nice mappings to abstractions of the scripting language (collections, iterators, closures, etc) -- but they are not exposed as such. I'd much prefer straight C to a bastardized Lua.

GRUB? (1)

rrohbeck (944847) | more than 3 years ago | (#36173074)

So what happened to Lua support in GRUB? I saw some pretty nifty things done with that.

Re:GRUB? (0)

Anonymous Coward | more than 3 years ago | (#36185314)

What the fuck? Seriously?
Scripting language in a bootloader?
What next, a rewrite in VB6 or in VB.NET?

Domain-specific Languages (0)

Anonymous Coward | more than 3 years ago | (#36173116)

Nowadays you would design special languages for domains using languages to help you built such language like Xtext, Xpand/Xtend. You use templates to generate Java, C or assembler code. This is more a stacking approach than an embedding approach. If I am not mistaken C# has some embedding features. Also tools like MPS support that direction. Even though the embedding method results in less developed DSLs than the top-down approach.

Yea its called.. (0)

Anonymous Coward | more than 3 years ago | (#36173212)


Lisp (2)

betterunixthanunix (980855) | more than 3 years ago | (#36173328)

Why jump through so many hoops to get an embeddable language, when you could use the same language to write the "host program" as you use to write scripts for it? Lisp is an example of such a concept: compiled Lisp programs still have the ability to interpret Lisp code, and hence the most sensible scripting language for a programming written in Lisp is Lisp itself. Lisp is not the only language with this property, but it is one of the most prominent examples of such a language.

Re:Lisp (0)

Anonymous Coward | more than 3 years ago | (#36173644)

In less than #!140

Re:Lisp (0)

Anonymous Coward | more than 3 years ago | (#36173658)

The purpose of Lua is to provide a lightweight scripting environment for applications written in C/C++. In that case your suggestion would be to embed C code in a C program, which isn't very useful as it wouldn't have any of the advantages of embedding with Lua such as faster compilations and not being C.

Oh, and do you want to know of another language that can have programs that interpret code written in the same language? Lua. It's even mentioned in TFA.

Re:Lisp (4, Informative)

Darinbob (1142669) | more than 3 years ago | (#36173942)

Because the host language may be a conventional language, designed to compile efficiently, run in small amounts of memory, has speed critical sections, etc. The embedded language on the other hand is used for doing stuff that wasn't thought of in the design phase and where it's simpler to add a small script than to add a new feature to the program. Generally what happens is that you're tying together a high level language with a low level language.

For instance, take a boot loader. It is written to fit into a small area of flash or just a few blocks of mass storage. So it needs to be small and tight. In manufacturing you'd like to run a set of tests and the boot loader provides some simple commands to call that test various features. But maybe you want a particular test that the built in commands won't do, for example a stress test of memory. The default method is to have development write a new command, check it into source code control, have QA run through it, approve it with the release schedule, then release it to manufacturing. Alternately one could just write a quick script and use that instead if the boot loader supported a scripting language, and manufacturing could manage this themselves without begging for R&D resources. Every time a bug occurs in the process someone could write a new regression test using the scripting language to check for the bug and add the script to a growing collection.

Now if we could somehow get fast/efficient/tiny host programs written in Lisp (which is indeed possible) and we manage to find a collection of developers who understand it (which is harder), then it makes sense to have the scripting language the same as the host language. But in the mean time most host programs are written in things like C or C++. Even if you have a high level interpreted language like Java that you wrote your host program in, it is very difficult to parse/interpret textual Java programs.

Even if your host program is in Lisp it will be difficult to isolate any textual Lisp scripts from the host program itself; you don't want a buggy script to cause havoc with the host program. Most Lisp systems I've seen that allow extra scripting basically use "eval" which is not very safe. So you still need to put up a solid wall with a well defined set of methods to allow a script to run in its own sandbox while interacting occasionally with the host program.

Re:Lisp (0)

Anonymous Coward | more than 3 years ago | (#36174018)

like a metacircular evaluator?

Re:Lisp (1)

Anonymous Coward | more than 3 years ago | (#36178712)

"The default method is to have development write a new command, check it into source code control, have QA run through it, approve it with the release schedule, then release it to manufacturing. Alternately one could just write a quick script and use that instead if the boot loader supported a scripting language, and manufacturing could manage this themselves without begging for R&D resources."

This means that your development/release process is broken with respect to your business needs, not that you need a nifty embedded scripting tool. That whole "ad hoc script added by the end user" is how we wind up with lame websites performing mission critical functions with 1000s of lines of totally unreviewed javascript.

Culture (0)

Anonymous Coward | more than 3 years ago | (#36180552)

One word: Culture

To flesh that out. Most cultures prefer infix notation. If you're targeting highly technical people as in Autocad, then embedding Lisp might work. Might. Even if the customers are as technical as Autocad users, they might still resist.

Re:Culture (1)

WNight (23683) | more than 3 years ago | (#36182168)

We have a prefix-form for math, observe:

add 3, 5, 10
multiply 2, 2, 2, 2, 2

Math is really the only thing we use infix for and if you discuss it using prefix operation, as above, nobody has a problem understanding, and therefore getting how lisp works. (add-this-list 1 2 3)

LISP's lispisms really only bug other, often narrowly skilled, programers who can't imagine another way to do anything. To someone learning from scratch it's so simple its oddity is of little importance.

Well, huh... You don't say. (1)

VortexCortex (1117377) | more than 3 years ago | (#36173344)

So, if you design your DSL based on the D, it will effect the S of your L?

Who'da thunkit?

Embeddibility? (1)

Mentally_Overclocked (311288) | more than 3 years ago | (#36173358)

After reading the headline, am I the only one that thought embedded in the context of microprocessors instead?

Re:Embeddibility? (0)

Anonymous Coward | more than 3 years ago | (#36174080)

Yeah, I thought about natural languages and processors.

Like... personally I thought that German was designed with embedded systems in mind.

Re:Embeddibility? (0)

Anonymous Coward | more than 3 years ago | (#36178598)

I honestly wasn't aware there were other uses of the term in the context of computing.

Self-morphing languages (2)

oldhack (1037484) | more than 3 years ago | (#36173852)

What's the track record for self-morphing languages like Lua and Forth? I just skimmed a Scala book, and it scares me.

I can see niches like producing DSLs, but smells like bad news for general application/system programming.

Tcl (2)

jdwoods (89242) | more than 3 years ago | (#36174072) [] says: "Tool Command Language (Tcl) is an interpreted language and very portable interpreter for that language. Tcl is embeddable and extensible, and has been widely used since its creation in 1988 by John Ousterhout. See [] for more info." Another good source of information on Tcl is []

Tcl functions well as glue between applications. Some folks know Tcl but call it "Expect" and may not realize Expect is simply Tcl plus an extension. Another extension, Tk, provides GUI features and is so powerful and popular that it's commonly used from other languages. Bindings exist for several other languages, including Ada (called TASH), Perl, Python (called Tkinter), Ruby, and Common Lisp.

Tcl is used by many people and companies (large and small). Cisco network gear uses embedded Tcl for automating tasks. Oracle uses Tcl for automating testing. The Fortune100 company where I work (but I am not a spokesman, so I won't name them) pays me to write and maintain an application written in Tcl to process payments for many thousands of customers totaling millions of dollars every day for payment through banks and the Federal Reserve.

Tcl is FOSS, but a very popular build is ActiveTcl from ActiveState. []

fanwagon (1)

Aighearach (97333) | more than 3 years ago | (#36174510)

There are lots of peeps pushing FFI, it's a bunch of crap IMO. They never seem to have any reason except portability, and they always try to hide or deny that it is slow and not very powerful.

There are already so many C libs, actually doing anything with it requires re-inventing all your wheels, and the wheels to make those wheels.

If C is too hard, you're not going to be doing embedded stuff long enough to get good with FFI anyways.

Entire article in three sentences (0)

Anonymous Coward | more than 3 years ago | (#36174592)

1) It must have an eval function or something like that to turn source into executable code.
2) It must be a library, so the host can load it.
3) Global variables must be accessible, but local variables must not be.
The article gives no argumentation for these.
Without going so far as to judge the claims, I think I can safely say that reading this comment instead of the article will save time.

Really you need to design your application round (1)

Anonymous Coward | more than 3 years ago | (#36175608)

Really you need to design your application round your embeded scripted language. If added as an afterthought it becomes a "lowest common denominator" interface between your application's type model and the embeded language's.

tcl (1)

drolli (522659) | more than 3 years ago | (#36176744)

i know, that will be hard to swallow, and tcl is not one of the , favourite 5 programming languages of mine, but its stable, easy to embed, has clear interfaces, and is leightweight. What else do you need?

Re:tcl (1)

dublin (31215) | more than 3 years ago | (#36245818)

Tcl is indeed an excellent choice for many embedded applications. It's not as trendy as many of the newer languages, but it is an awesome tool for getting a lot done with a very small amount of code (some joke that it stands for "Try Coding Less"), and with very minimal overhead.

Lua's certainly intriguing, and I might look at it for future projects, but the vast majority of what I've had my teams build in the past few years is Tcl (if small size is important), or Python (if it's less important).

Like Python, Tcl approaches executable pseudocode - it's almost always trivial to read a Tcl program and be able to follow it, unlike many other languages. The value of this for things that you (or others) have to maintain and/or extend over time is incalculable...

Tcl's biggest ding right now is that it doesn't have a "popular and modern" snazzy web app platform right now, although AOLserver has great integrated-from-the-ground-up Tcl support, and is definitely one of the apps that shows that Tcl can scale up as well as it scales down....

Nu has it all (0)

Anonymous Coward | more than 3 years ago | (#36177830)

The should check out Nu: []
Full interoperable with Objective-C and thus C/C++.

Check for New Comments
Slashdot Login

Need an Account?

Forgot your password?