30 December, 2010

Introducing EurekaLog 6.1

We made a present for our customers at New Year 2010-2011 by releasing EurekaLog 6.1. So... what's new?

Q: What is new in EurekaLog 6.1?
A: EurekaLog 6.1 is exactly the same as EurekaLog 6.0 (6.0.25 is the latest available version), except one "little" detail: we've added support for new compiler from EurekaLog 7.0 (which is in development right now).

Q: Why is it 6.1? Why not 6.0.26?
A: We do only bug fixes in 6.x branch, so we get 6.0.10, 6.0.20, 6.0.25, etc. Each new version (like 6.0.xx) contains bug fixes, but no new features. Since we've added a major new feature - we decided to increase a version number, so new version became 6.1 - indicating addition of a major new feature.

Q: Is it safe to upgrade? Am I affected?
A: Don't worry, 6.1.xx behaves as 6.0.xx by default. You can safely upgrade and no new changes will affect you (except the usual bug fixes). New compiler have no effect, until explicty enabled.

Q: What's so great about new compiler?
A: In recent 3 months there were many reports about problems with EurekaLog and debug information (i.e. mapping between code and human-readable names of units, methods, etc). After investigation we've discovered that there appears linker bugs (usually on large projects), which prevents EurekaLog from proper functioning. Also, there are some internal limitations of EurekaLog for large projects1.

What does it mean? It means that for some large projects EurekaLog can't show you a proper call stack because of wrong debug information, which is usually caused by linker's bugs.

Well, I'm a bit surprised that no one is ever reported such issues before (at least not while I was working for EurekaLab s.a.s.). So these last months were really a hit on this.

OK, so the new compiler from EurekaLog 7 have a new architecture and a new code (it's re-written almost on 80%) and it is able to solve these problems (along with help of new format of debug information).

Another advantages are:
  1. Increased speed. New compiler should be (theoretically) faster than old, since we've reduced amount of work with strings.
  2. Reduced memory consumption. New compiler should be (theoretically) more conservative about memory usage, since we're working on since mapped file, instead of parsing it into separate strings.
  3. Increased stability. Any issues with compiler itself does not affect IDE, since compiler is now run into separate process.
  4. Increased detalization. New compiler is able to store routines without line numbers. Old compiler can't do this. Usually this doesn't matter anyway, since we always recommend to enable "Use Debug DCUs" option.
Items 2 and 3 allows you to use EurekaLog for very large projects, which wasn't possible before (because of shared virtual address space with IDE and, thus, hitting a memory limits).

Word "theoretically" means that we think so, but we don't have any exact estimates yet. So, if you're owning a large project (say, 10+ MB .exe file or 15+ MB .map file) - please, send us (even very approximate) details about how fast (or slow!) compilation is with old and new compiler.

Q: Why add compiler from EurekaLog 7 to EurekaLog 6? Why not just fix issues in EurekaLog 6? Or just wait for EurekaLog 7 release?
A: Because it's impossible to fix these issues with old logic of compiler (there are much technical details here, I won't going to discuss them). To solve these issues in EurekaLog 6, one needs to almost re-write the compiler - pretty much the same job as was already completed in EurekaLog 7 (compiler upgrade in EurekaLog 7 wasn't because of these issues - we just improved code. Solving these issues was a side-effect).

So why just not port code from EurekaLog 7 to EurekaLog 6? Well, we tried this, but it's not easy too, since there are a lot of changes (also remember that we dropped support for Delphi 3, so we used new features a lot).

Why not just leave this as is, claiming that customers should wait for EurekaLog 7? I'm not saying that EurekaLog 7 have a long road before release, but I'm saying that we care about our customers, so we've decided to release intermediate EurekaLog 6.1, which solves some major problems of our customers.

Thus, the only solution left is to include new compiler as an option to EurekaLog 6.

Q: When I should use new compiler?
A: If you have small/medium project and never suffer from weird call stacks - then you can just ignore it. Don't use new compiler (which is default, BTW). Otherwise - try it. New compiler may help you to solve your issues.

Q: How to use it?
A: Very simple. Just go to Project / EurekaLog options, select Build options tab and enable Use EurekaLog 7 compiler checkbox:

Q: What about command line compilation and build automation?
A: That is possible too. You can find new compiler in your EurekaLog folder under sub-folder which corresponds to your IDE. For example:
C:\Program Files\EurekaLab\EurekaLog 6\Delphi15\el7c.exe
The same .exe file can be used for both Delphi and C++ Builder - just be sure that you pick a correct version (in case if you have several different versions of IDE installed).

You can use this file exactly as the old ecc32.exe/emake.exe. Except there are few additional options:

  • --el_output=el6 - instructs compiler to produce debug information in EurekaLog 6 format. Mandatory. You must add this switch.
  • --el_mode=delphi or --el_mode=builder - switches between Delphi and C++ Builder project. Mandatory. You must add this switch.
  • --el_path=folderpath - folderpath is full absolute path to EurekaLog folder. This is optional switch, but we strongly recommend to specify it. For example:
    "--el_path=C:\Program Files\EurekaLab\EurekaLog 6"
  • --el_gui_error - show errors in message boxes instead of writing them to the console. Optional. Usually you want to add this, if you run compiler from IDE or other GUI tool. You probably don't want to specify it, if you run compiler from console or automated build script.
  • --el_target=filename - specifies output file name. Optional. By default EurekaLog tries to get this from project's options (which is not always possible). It's not needed for --el_alter_exe switch, since output file name can be specified inside --el_alter_exe switch itself.

Q: I don't want to use experimental compiler from a EurekaLog 7, which doesn't even have a stable release!
A: First I want to highlight that EurekaLog's code remain the same. New compiler doesn't affect the code. It's still old good and tested 6.0.25. New compiler affects only additional debug information. So, if you have problems with debug information, new compiler may solve it. I.e. it's not worse than it's already is. Surely, you may not use new compiler, if you don't need to.

Yes, new compiler comes from unstable version of EurekaLog 7. However, we've tested a new compiler quite strict. Sure we may miss something, but it's quite stable now.

Of course, you shouldn't just enable this option and throw your application to your customers. You need to do a few simple tests:
  • Try just to compile your application with new compiler - will it be successfull? New compiler contains additional asserts, which trigger on unsupported cases.
  • Take a known issue with bug report and check this issue against new compiler - will a new bug report be better than old?
  • You can place a button, which raises exception somewhere deep inside your code. Will a bug report be detailed enough?
If these tests will pass - then you can safely use new compiler. If not - please, contact us, so we can improve our product!

Thank you for all your valuable feedback and I wish you a happy New Year!

1 - for example, EurekaLog uses signed word (2 bytes) as offset between code from neighbour lines. Usually this is not a problem, since machine code goes in the same order as source (indeed, many people even rely on this undocumented behavior), so you rarely see values larger than 128 - not even saying about 32768!

However, we've found that in some (rare) cases code may go in really wild order. So if you wave a unit larger than 32768 lines AND such thing happens (I don't know if this is another linker bug or planned behavior) - EurekaLog won't be able to handle this. New debug format of EurekaLog 7 uses up to 4 bytes for offsets, so these cases are covered. However, old debug format is still limited to 2 bytes offsets.

There may be 1 or 2 other but similar issues - I don't really remember them now.