MonetDB Feb2010-SP2 release

Release "Feb2010-SP2" consists of the following updated components

Name	Version	CVS module	CVS branch	Note/Comment
MonetDB Common	1.36.5	MonetDB	Feb2010
MonetDB Clients	1.36.5	clients	Feb2010
MonetDB4 Server	4.36.5	MonetDB4	Feb2010
MonetDB4/XQuery	0.36.5	pathfinder	Feb2010
MonetDB5 Server	5.18.5	MonetDB5	Feb2010
MonetDB5/SQL	2.36.5	sql	Feb2010
MonetDB5/SQL/GIS	0.16.5	geom	Feb2010
MonetDB Java	1.36.5	java	Feb2010
MonetDB Testing	1.36.5	testing	Feb2010	(developers only)

NOTE:
Before upgrading to the latest "Feb2010-SP2" release from a release of MonetDB older than the Nov2008 release, please make a dump (SQL, XQuery) of your database, and rename/move/backup the database directories (dbfarm, sql_logs, xquery_logs). After the upgrade, restore (SQL, XQuery) your database, again. Once you have checked and confirmed the correctness of the restore, you can delete the above renamed/moved/backed-up old database directories.

When upgrading from a release of MonetDB older than the May2009 release, you need to move your write-ahead log following the "recommended" instructions for SQL and/or XQuery.

Contents

• Feb 2010 SP2 bug-fix release
• Feb 2010 SP1 bug-fix release
• Feb 2010 feature release
  • Packaging
  • MonetDB Common
  • MonetDB Server 5
  • SQL
  • XQuery
  • Client Packages
  • Bug fixes

Feb 2010 SP2 bug-fix release

This is the second bug fix release of the Feb2010 version. This version can be installed "on top of" the Feb2010 release. The database format is fully compatible, so there is no need to do a dump and restore. (Although it is *always* a good idea to make backups.)

Some of the bugs that were fixed:

• Fixed a bug that could cause a crash when string BATs are combined. (SF bug 2947763.)
• Fixed a potential file leak: under certain conditions, files in the database might not get deleted when they should (they would be deleted when the server restarts).
• Fix regression introduced in Feb2010-SP1 causing UDP connections to malfunction, in particular affecting the stethoscope tool.
• Documented the database option in the DSN in the Perl interface. Also incremented the version number of the Perl interface.
• Fixed a problem in the PHP interface where under certain circumstances not all data from the server was read, leading to a limited number of rows being returned and other weird behavior. This fixes SF bug 2975433.
• If mclient is called with argument -? or --help, exit with exit code 0. This fixes SF bug 2956574.
• fixed bug #2983773 "SQL: minimal optimizer pipe unstable" https://sourceforge.net/tracker/index.php?func=detail&aid=2983773&group_id=56967&atid=482468
• Fixed a problem preventing the opening of remote documents in a doc("http://...") query.

The following bug tracker items were fixed since the Feb2010-SP1 release:

• [ 2942744 ] BATpropcheck fails on C Mapi usage
• [ 2947763 ] SQL: insert into + select on same table
• [ 2956574 ] 'mclients --help' returns 255
• [ 2964247 ] Merovingian: crash upon client connect from remote host
• [ 2971516 ] pci->fcn = 0 in malAtomProperty: segfault
• [ 2975433 ] PHP Native client occasionally truncates large result sets
• [ 2982104 ] monetdb status throws warning with large num of dbs
• [ 2983773 ] SQL: minimal optimizer pipe unstable

Feb 2010 SP1 bug-fix release

This is a bug fix release of the Feb2010 version. This version can be installed "on top of" the Feb2010 release. The database format is fully compatible, so there is no need to do a dump and restore. (Although it is *always* a good idea to make backups.)

Some of the bugs that were fixed:

• Fixed bug in UDP stream creation causing UDP connections to already bound ports to be reported as successful. This fix is known to cause a regression rendering stethoscope useless.
• Fixed a race condition that could occur if many clients were connecting and disconnecting.

Bugfixes for Merovingian:

• Under certain conditions, monetdb commands status and get could fail to properly return all or any data and exit with a deserialise error instead.
• Connecting from a client that has no name setup in DNS could cause merovingian to crash, bug #2964247.

The following bug tracker items were fixed since the Feb2010 release:

• [ 2907463 ] M5: pma.mx fails to compile on Windows
• [ 2914578 ] Test for -lcrypto with -lssl not working
• [ 2947747 ] name default_pipe is misleading
• [ 2960174 ] compiling fails with --with-pcl
• [ 2960781 ] M5: potiential SEGFAULTs in mal_interpreter
• [ 2963782 ] Patch for Mx Segfault

Feb 2010 feature release

Fedora

We now package for Fedora 11 and 12.

• Various concurrency bugs were fixed.
• Various changes were made to run better on systems that don't have enough memory to keep everything in core that was touched during query processing. This is done by having the higher layers giving hints to the database kernel about future use, and the database kernel giving hints to the operating system kernel about how (virtual) memory is going to be used.
• Fixed bug in mergejoin implementation. This fixes bug #2952191.
• Added support for compiling on Windows using the Cygwin-provided version of flex.
• Fix compilation issue when configured with --with-curl. This fixes bug #2924999.
• Added implementation of MT_getrss() for Solaris. This yields in the kernel knowing about its (approximate) memory usage to try and help the operating system to free that memory that is best to free, instead of a random page, e.g. the work of the vmtrim thread.
• Implemented a "fast" string BAT append: Under certain conditions, instead of inserting values one-by-one, we now concatenate the string heap wholesale and just manipulate the offsets. This works both for BATins and BATappend.
• Changed the string heap implementation to also contain the hashes of strings.
• Changed the implementation of the string offset columns to be dynamically sized.

• The MonetDB 5 code base underwent a series of small/medium changes:
  • Solving name conflicts with external libraries in mal_box.
  • Dependency graph generation in MAL debugger fixed.
  • Code hardening against out of memory errors and based on Coverity checks.
  • Recycler improved to deal with SQL plans from different sessions.
  • Profiler extended to report the argument types, user, and thread id.
  • MAL interpreter reports an event before and after the instruction.
  • Dataflow also allowed for updates on temporary BATs.
  • Reorder optimizer better respects the dataflow.
  • All update instructions return their target to mark the dataflow.
  • Clean up of (bat)calc module.
  • Packing pieces together simplified and sped up.
• The MAL interpreter has been extended with an operator admission scheme, which is active during parallel execution. It blocks threads if the total amount of memory needed for the operator can not be claimed. Only if there is one operation left to execute, it won't block. The admission level is controlled by a threshold, which is set to 90% of the physical memory.
• The ilike[u]select operations has been included in the repertoire recognized by the mergetable optimizer for push through of selections.
• The joinpath optimizer has been extended with recognition of the leftjoin. This way, series of leftjoin operations can be optimized by looking at the smallest starting point.
• Introduced a dictionary encoding option to the optimizer pipeline. Encoding is initiated with a SQL call compress(tablename).
• Removed the ability to redirect to other running databases in the same dbfarm. This functionality has been taken over completely by merovingian, and only results in confusement these days, bug #2891191.
• Added --enable-console configure argument, defaulting to 'yes' for now. Disabling the server console increases security by avoiding local access exploits. This is not the default since our Testing setup cannot deal with a console-less server yet.
• Report detected amount of main memory and cpu cores in output of `mserver5 --version`.

• Fixed a bug in the Mapi libary when sending very large queries to the server.
• Implemented BLOB and CLOB support in ODBC driver.
• Fixed crash upon connect to a server that is under maintenance but has the requested language (scenario) not loaded.
• Fixed bug in the ODBC driver where a non-ASCII character at the end of a string caused an error when the string was converted to wide characters.
• mclient timer information (-t option) is now written to standard error instead of standard output. This makes getting timer information easier when there is a large amout of regular output (which can be redirected to a file or /dev/null).
• Implemented new function mapi_fetch_field_len() which returns the length (excluding trailing NULL byte) of the field returned by mapi_fetch_field().

The following bug tracker items were fixed since the previous "Nov 2009" release:

The detailed list of changes can be found in the source code CVS logs.