|
|||||||
|
|
|
|||||
|
|
|||||||
Please, don't get too excited (or too overly critical!) This "project" is still only in the research stage. No code (by myself) has been written as yet. I'm still struggling with this PC and trying to get my old Amiga stuff copied over so I can run it on UAE.
Now, the real question: why?
Because I can't think of a better project. I think Portable Binaries (PB's) are the most important thing to come along in a long time. The collective excitement over Java is proof of that. However, I think Java is the wrong way to do it.
But more specifically, PB's would be especially of benefit to the Amiga. The Amiga is fragmenting, what with Amiga Inc essentially trying to kill it, and third party companies trying to either sustain the existing technology or create something totally new. Personally, I want to see AmigaOS live on. But, to live on which processor? 68K? PPC? Something else? Well with PB's it doesn't matter. Wouldn't it be nice to run your powerful Amiga software on your Amiga, on your new PPC card at full speed, and also on your PC clone under UAE? Lets see any other OS/computer do that, even under Linux, without recompiling and porting headaches.
Here are some links to other resources relating to portable binaries.
I welcome any positive comments or help, please feel free to email me or ICQ me (26669745).
I've been seeing mixed opinions on the subject of portable binaries. Some people just don't see the point in it, some think it's just not that useful, others say it's too hard or impossible to do in a way that optimizes for complex modern CPU's, and finally the visionary people who think it's the best thing since sliced bread.
First of all, the usefulness question: All I have to say is, look at Java. If seamless portability wasn't important, Java wouldn't be popular. Especially since Java has a performance penalty.
Second, feasibility: It can be done. There's ANDF, but more importantly there's the SDE and Juice systems. Juice is particularly interesting because it's really small.
The biggest question, then, is how efficient could portable binaries be? I've been told that optimizing for modern CPU instruction sets is very difficult, particularly new things like MMX, SIMD and AltiVec/Velocity Engine; that the primitives in the portable binary format would have to be upgraded every time a new CPU came along; etc.
But think about this for a moment. Think about how important portability is, both for users and developers. OK so maybe the programs won't be perfectly optimized on every CPU there is... but at least they will run! And the chances of your software taking advantage of new CPU features are better because the code generator can be upgraded, whereas today you have to upgrade each program you have (if there even is an upgrade). And the developers don't need to build and distribute versions of their software for every CPU out there.
Let's compare to Java. While it hasn't taken over like Sun thought it would, it IS gaining popularity. The overhead of Java is lessening, what with faster CPU's and better JIT's. So you may ask why bother with portable binaries when Java is here already? Because Java uses a different method -- it really isn't a portable binary format; rather it is a binary format for essentially a made up CPU which is then interpreted.
In order to get good speed, Java can employ a Just In Time (JIT) compiler. In this system, the Java program is interpreted one time and the resulting native code is run from then on. Seems like a great idea, but, it suffers from exactly the same problem that many people attribute to portable binaries -- only worse.
Since Java byte codes are a machine language, much of the higher-level semantics of the source language are lost. General symbolic operations become very specific, literal operations (optimized for the Java byte code). In other words, it's hard to optimize Java. Much harder than it would be to optimize portable binaries. But somehow they're doing a pretty good job of it.
So it comes down to this: if they can make Java run that fast, then there's no reason portable binaries couldn't be as fast or faster. And probably with a lot less effort and overhead.
So... why isn't everybody using it?
Because current implementations have limitations. There's been lots of different solutions to the problem, each with advantages and limitations. The most promising I think was/is SDE/Juice. Here's why it hasn't taken off:
So, in order to make this great technology work, we need to (in case you can't figure it out from the above...:):
Well, as is typical for me to do (must be the mapper in me), I am finding a lot of little bits and pieces of things that others have done which could apply to this project. While I probably wouldn't take anything wholesale and use it, it does seem logical to try and leverage off what others have already done (not to mention they probably know more about what they're doing than I do).
So, here are the ingredients thus far (in approximately the order in which they would be used in the process of compiling an APB program):
| Language / front end | Abstract Syntax Definition | "Portable Assembler" / back end | Run-time | |
| C | -> IR | -> | ||
| Oberon | -> IR | -> ASDL | -> C-- -> APB | -> Code Generator |
| Java | -> IR | -> |
At the top, you have your language. C is the first goal here, although `C (Tick-C) might be just as easy. `C allows you to create specialized, dynamically created code. It currently uses its own internal methods, but with SDE this wouldn't be necessary. `C would allow the programmer to take advantage of SDE's functionality to create optimized code. The SDE system should allow "hooks" for such languages and tools to directly manipulate the code generation. `C uses something called vcode internally, which uses an idealized RISC-like virtual machine which is converted "in place" to target machine code on the fly. Advantages are that it's very fast and uses little memory.
"IR" is the Intermediate Representation of the compiler front end, which could be different for different compilers. ASDL Defines how the IR can be mapped to an ASDL, and the data structures and procedures to manipulate the ASDL. This will make it easier to use different front ends and languages.
At this point, optimizers could be employed if they use ASDL. ASDL enables an interchangeable component based compiler system. Hopefully this will enable the APB system to be more popular as other languages are connected to the front end.
Finally, the back end takes the ASDL and converts it to an SDE file. Despite how it looks, there's only two passes just like a normal compiler. The front end is the specific language parser, its IR and the ASDL stuff. The back end is the ASDL, SDE compiler and APB "linker". I am still debating the usefulness of C--; would it make a good basis for the IR or is too much high-level Symantec information lost? It is worth studying though, just to make sure that whatever IS used is capable of the same functionality. The C-- run-time extensions however are useful without C-- the language.
The last part of the system is the run-time environment. This is where the APB is converted on the fly to native code. More optimizations occur at this point. Also there is the C-- run time which basically provides "backward hooks" so that debuggers and high level language systems like garbage collectors and exception handlers can get the low level information they need, and be able to associate it with the high-level constructs. The run time code generators of `C, SDE/Juice and C-- are all under investigation.
I've tentatively selected the lcc retargetable compiler as the C compiler for APB. In theory, I plan to create a new back-end that produces APB's, and take the normal back-end code generation and work that into the APB loader. I believe there is a code generator for 68K already, and hopefully there is a PowerPC one as well.
LCC seems somewhat popular. There is `C, based on LCC, and LCC-win32 which is a full development suite not unlike that stuff from MS. And there's an (expensive) book on it, but also some rather useful documentation in the lcc-win32 documentation on the inner workings of lcc.
There are portability concerns:
The sizes of datatypes may (will) change from processor to processor. How to handle this? You could simply garantee the sizes and make the compiler/run time do whatever backflips necessary (for instance if a program used 64 bit math and ran on a 16 bit CPU the math operators would call necessary math routines..). Or you could require programs to explicitly check the sizes (sizeof() in C) and be smart about it -- but that means much software may not be immediately portable. A compiler may have options to specificaly set the sizes that the source code expects.
Endian-ness: This could be very annoying. If a program were compiled expecting one endian-ness but ran on a CPU with the other endian, every access to variables would need manipulating... This is the same problem AROS faces when trying to make it work with UAE.
| AROS | -> AmigaOS | -> UAE | -> QNX | -> PPC |
| -> AmigaOS | -> 68K |
"Code generation" to take advantage of 3DFX cards. 3DFX language could be targeted on the fly to different GFX cards, or to CPU code. Ideally, 3DFX primitives would be converted to API calls in a 3DFX library for a particular card; which could then in turn be inlined, thus resulting in a hardware banging program that's compatible with card that has a driver!
Code generation for DSP's.
This isn't directly related to APB but I thought of it just now: How about the ability to write PC programs that use the Amiga API through UAE? They would run from DOS or Windows but bring up Amiga/UAE windows... With AmigaOS being emulated but the program native, I wonder if the program's apparent speed would be greater or less than a Windows standard program?
Many functions could be inlined to save overhead. This inlining happens at run time, thus saving space in the APB file.
Instead of setting the library base in A6 and jumping to the function table in the library, and then another jump to the actual function, the library is "interrogated" for the final function pointer and this is used directly to call the function. Unfortunately many (most? all?) library functions will expect the library base in A6, so this still has to be done. In a system where the library itself is coded in APB this won't be needed, and in fact, the code of the function itself could be inlined! It may even be possible to inline BOOPSI method calls.
LD A6, DosBase; 16 bit instruction, 32 bit address
JP A6(Write); 16 bit instruction, 16 bit offset?
...
JP #xxxxxxxx; 16 bit instr, 32 bit long address
Replaced by:
LD A6, DosBase;
JP #xxxxxxxx;
Naturally, any programs that patch functions in libraries must be run first. There should be a priority system in place to ensure things get run in the right order, or to initiate code-regeneration for dependant code.
Registerization of function parameters, and intelligent variable-length argument lists. The later, specifically, refers to tag lists. Tag lists could be built on the stack, allocated into memory, or whatever, depending on the CPU being used.
Based on the arguments to a function, a specialized version could be generated optimized for those arguments. A very generalized function which may branch internally based on clearly defined arguments could be specialized at run time to bypass the checks.
Taking this dynamic code generation concept even further, it could possible to literally build a function statement by statement and then execute it, using a language like `C (or any language with similar extensions).
This combined with run-time code optimizations could result in very fast operations. Consider the process of redrawing everything in a window full of gadgets and images. There's a lot of code to go through -- running down the linked lists, BOOPSI calls, calculations, etc. This gets done every time you have to refresh the window, even if nothing changed. You could collect just the rendering calls and dynamically create a function with those calls in it. Then, passing this through the code generator, it gets optimized - some calls may even get inlined; certainly some arguments (rastports) could be kept in a register, saving time.
Continuously running profiler locates candidates for optimization, and proceeds to optimize those code sections. Information such as the number of clients of a library, the number of times it is loaded, etc, could be used to select what code would benefit the system most by being optimized. When a module is optimized it would then replace the existing code. One problem with this is that if the code is being accessed in some way, it can't be changed. For instance, if a function in a library had been in-lined into another library -- it couldn't be optimized without regenerating that library too. Simple references from other modules could be patched, however, even on the fly by using the run-time services described below.
Loop unrolling. Small frequently used loops could be unrolled on the fly.
Optimization weighting - tracks time spent on optimizing heavily used code, over a certain threshold and the native code is cached to disk to be loaded directly next time. It is likely that the OS, if coded in APB, would be cached in this way. Additionally, the user could force this option.
I've had ideas for an object oriented system for some years now. With APB, many of the problems I encountered could be eliminated. As well, new functionality is possible.
Full run time binding with the efficiency of static binding. Since APB resolves symbols at load time, this is now a no-brainer.
Autonomous objects. An object containing methods coded in APB, with a clearly defined interface, could be completely autonomous. It could move from task to task, processor to processor, system to system, across memory and network boundaries, be archived, bond with other objects, etc.
To get the full effects it may be necessary to have the OO system in the language (C++, Objective C, Oberon, ?) compiler.
Because of the nature of the SDE system, it is possible to have different levels of "scoping", each revealing a set of symbols to the executable - no more than necessary for the code to run. Thus, a lot of security comes with the system. At the lowest level, all symbols are available - system registers, all instructions, all memory areas, etc. At the highest level - probably where things like applets in web pages would run - there is a restricted symbol set which may not include, for instance, things like the OS file system calls, memory pointers (I.E., no absolute memory addressing), etc.
With this kind of scoping, much of the need for CPU level modes (Supervisor / User), memory protection, and expensive analysis of code for violations (like Java does) is not necessary.
There are some language features which require a close cooperation of a "normal" compiler's front and back ends. Exceptions, garbage-collection and debugging require information known only to each the front and back ends. Since APB separates these two functions, there needs to be a way to communicate the information across.
C-- is a language meant to handle this problem -- as it relates to the idea of being able to separate front and back ends to facilitate interchangeability of these components. However, it could be useful as the basis for the intermediate language of APB's, and at the least, the C-- run-time system is a useful addition to the APB run-time environment.
 | |||||||||
 Aric R Caley. / Greywire / webmaster@greywire.8m.com / HOME
|
|||||||||