lakesuperior_legacy/docs/release_notes.rst

=============
Release Notes
=============

1.0 Alpha 18
------------

*October 10, 2018*

This release represents a major rewrite of many parts of the application, which
took several months of research and refactoring. The main change is a much more
efficient storage engine. The storage module and ancillary
modules were completely rewritten in Cython to use the LMDB C API rather than
the LMDB Python bindings [#]_. Most of the stack was also modified to
accommodate the new interface.

Most of the performance gains are visible in the Python API. Further
optimizations would be more involved, including refactoring RDF serialization
and deserialization libraries, and/or SPARQL language parsers. That may be done
at the appropriate time.

Note that from this version on, Lakesuperior is distributed with C extensions
(the Cython modules). This requires having a C compiler installed. Most Linux
distributions come with `gcc <https://www.gnu.org/software/gcc/>`__. The
C sources generated by Cython are distributed with the package to avoid a
dependency on Cython. They are very large files. Adopters and most contributors
should not be concerned with these files.

New Features
~~~~~~~~~~~~

- New Python API objects (
  :py:class:`~lakesuperior.store.ldp_rs.lmdb_triplestore.SimpleGraph` and
  :py:class:`~lakesuperior.store.ldp_rs.lmdb_triplestore.Imr`) for simple and
  resource-efficient handling of sets of triples.
- New features in benchmark script:

  + Command line options that deprecate interactive input
  + Request/time graph plotting

Enhancements
~~~~~~~~~~~~

- New storage layer providing significant performance gains, especially in read
  operations. See :doc:`Performance benchmark results <performance>`.
- Test coverage has increased (but is still not satisfactory).

Bug fixes
~~~~~~~~~

Several pre-existing bugs were resolved in the course of refactoring the code
and writing tests for it:

- faulty lookup method involving all-unbound triples
- Triples clean-up after delete
- Other minor bugs

Regressions
~~~~~~~~~~~

- Removed ETags from LDP-RS resources. Read the
  :doc:`Delta document <fcrepo4_deltas>` for more explanation. This feature
  may be restored once clear specifications are laid out.
- Increased size of the index file. This is a necessary price to pay for faster
  retrieval. The size is still quite small: see :doc:`Performance
  <performance>` for details.

Other Significant Changes
~~~~~~~~~~~~~~~~~~~~~~~~~

- The ``fcrepo`` shell script, which launches the multi-threaded gunicorn web
  server, is now only available for Unix system. It would not work in Windows
  in previous versions anyways. Note that now this script is not in the
  ``$PATH`` environment variable of the virtualenv and must be invoked by its
  full path.
- Main LDP-RS data and index are now in a single file. This simplifed the code
  significantly. The previous decision to have separate files was made for
  possible scenarios where the indices could be written asynchronously, but
  that will not be pursued in the foreseeable future because not
  corruption-proof.
- Release notes are now in a self-standing document (this one) and are referred
  to in Github releases. This is part of a progressive effort to make the
  project more independent from vendor-specific features (unrelated from
  Github's recent ownership change).

1.0 Alpha 17 HOTFIX
-------------------

*May 11, 2018*

Hotfix resolving an issue with version files resulting in an error in the UI
homepage.

1.0 Alpha 16
------------

*April 28, 2018*

This release was triggered by accidentally merging a PR into master instead of
devleopment, which caused CI to push the a16 release, whose name cannot be
reused...

In any case, all tests pass and the PR actually brings in a new important
feature, i.e. support for multiple RDF serialization formats, so might as well
consider it a full release.

1.0 Alpha 15
------------

*April 27, 2018*

Alpha 15 completes version handling and deletion & restore of resources, two
key features for the beta track. It also addresses a regression issue with
LDP-NR POSTs.

All clients are encouraged to upgrade to this last version which fixes a
critical issue.

New Features
~~~~~~~~~~~~

- Complete bury, resurrect and forget resources
- Complete reverting to version (#21)

Enhancements
~~~~~~~~~~~~

- Dramatic performance increase in GET fcr:versions (#20)
- Refactor and simplify deletion-related code (#20)
- Minimize number of triples copied on version creation
- Complete removing SPARQL statements from model and store layout; remove
  redundant methods

Bug Fixes
~~~~~~~~~

- LDP-NR POST returns 500 (#47)

Other Changes
~~~~~~~~~~~~~

- Add PyPI package badge in README

Acknowledgments
~~~~~~~~~~~~~~~

Thanks to @acoburn for reporting and testing issues. 

1.0 Alpha 14
------------

*April 23, 2018*

Alpha 14 implements Term Search, one of the key features necessary to move
toward a Beta release. Documentation about this new feature, which is available
both in the UI and REST API, is at
`<http://lakesuperior.readthedocs.io/en/latest/discovery.html#term-search>`__
and in the LAKEsuperior term search page itself.

This release also addresses issues with Direct and Indirect Containers, as well
as several other server-side and client-side improvements. Client making use of
LDP-DC and LDP-IC resources are encouraged to upgrade to this version. 

New Features
~~~~~~~~~~~~

- Term search (#19)
- Allow creating resources by providing a serialized RDF bytestring (#59)

Enhancements
~~~~~~~~~~~~

- Upgrade UI libraries to Bootstrap 4 
- Write tests for Direct and Indirect Containers (#22)

Bug Fixes
~~~~~~~~~

- LDP-RS creation with POST and Turtle payload results in a LDP-NR (#56)
- Cannot add children to direct containers (#57)


Acknowledgments
~~~~~~~~~~~~~~~

- Thanks to @acoburn for reporting issues. 

1.0 Alpha 13
------------

*April 14, 2018*

Alpha 13 addressed a number of internal issues and restructured some core
components, most notably configuration and globals handling.

New features
~~~~~~~~~~~~

- Report file for referential integrity check (#43)
- Support PATCH operations on root node (#44)
- Version number is now controlled by a single file
- Version number added to home page

Enhancements
~~~~~~~~~~~~

- Better handling of thread-local and global variables
- Prevent migration script from failing if an HTTP requests fails
- Light LMDB store optimizations

Bug fixes
~~~~~~~~~

- Removed extraneous characters from ``anchor`` link in output headers (#48)

Other changes
~~~~~~~~~~~~~

- Added template for release notes (this document). This is not a feature
  supported by Github, but the template can be manually copied and pasted from
  ``.github/release_template.md``.

Notes & caveats
~~~~~~~~~~~~~~~

- Custom configurations may need to be adapted to the new configuration scheme.
  Please look at changes in ``lakesuperior/etc.defaults``. Most notably, there
  is now a single ``data_dir`` location, and ``test.yml`` file is now deprecated.

Acknowledgments
~~~~~~~~~~~~~~~

Thanks to @acoburn for testing and reporting several isssues.


1.0 Alpha 12
------------

*April 7, 2018*

Alpha 12 addresses some substantial enhancements to the Python API and code
refactoring, additional documentation and integration.

Features
~~~~~~~~

- Integrate Travis with PyPI. Builds are now deployed to PyPI at every version
  upgrade.
- Allow updating resources with triple deltas in Python API.

Enhancements
~~~~~~~~~~~~

- Streamlined resource creation logic, removed redundant methods.
- Allow PUT with empty payload on existing resources.

Bug Fixes
~~~~~~~~~

- Referential integrity did not parse fragment URIs correctly.

Other
~~~~~

- Added documentation for discovery and query, and Python API usage examples.

1.0 Alpha 11
------------

*April 4, 2018*

Alpha 11 introduces some minor documentation amendments and fixes an issue
with the distribution package. 

Features
~~~~~~~~

None with this release.

Enhancements
~~~~~~~~~~~~

- Documentation improvements.

Bug Fixes
~~~~~~~~~

- Issue with config files in wheel build.

1.0 Alpha 10
------------

*April 3, 2018*

Alpha 10 introduces a completely revamped documentation and integration with
setuptools to generate Python packages on PyPI. It incorporates the unreleased
alpha9.

Features
~~~~~~~~

- Translate all documentation pages to .rst
- Several new documentation pages
- Translate all docstrings to .rst (#32)
- Documentation now automatically generated by Sphinx
- Setuptools integration to create Python wheels

Enhancements
~~~~~~~~~~~~

- Moved several files, including default config, into lakesuperior package
- Reworked WSGI (gunicorn) server configuration, now exposed to user as .yml
  rather than .py
- Made most scripts non-executable (executables are now generated by setuptools)
- CI uses setup.py for testing
- Web server no longer aborts if STOMP service is not accessible

Bug Fixes
~~~~~~~~~

None with this release.

Other
~~~~~

- Documentation now available on `<https://lakesuperior.readthedocs.io>`__ and
  updated with each release
- Python package hosted on `<https://pypi.org/project/lakesuperior/>`__.
  Please make sure you read the updated install instructions.
- Switch semantic version tag naming to a format compatible with PyPI.


1.0 Alpha 8
-----------

*March 26, 2018*

Alpha 8 introduces a migration tool and other enhancements and bug fixes.

Features
~~~~~~~~

- Migration tool (#23)
- Referential integrity checks (#31)

Enhancements
~~~~~~~~~~~~

- More efficient and cleaner handling of data keys (#17)
- Enhanced resource view in UI
- Simplified and more efficient PATCH operations
- Zero configuration startup
- More minor enhancements

Bug Fixes
~~~~~~~~~

- STOMP protocol mismatch
- Missing UID slash when POSTing to root (#26)
- Running out of readers in long-running processes

Other
~~~~~

- Travis and Docker integration


1.0 Alpha 7.1
-------------

*March 9, 2018*

1.0 Alpha 7
-----------

*March 6, 2018*

This is the first publicly advertised release of LAKEsuperior.

Some major features are missing and test coverage is very low but the
application is proven to perform several basic operations on its own and with
Hyrax 2.0.

1.0 Alpha 6
-----------

*February 28, 2018*

1.0 Alpha 5
-----------

*February 14, 2018*

1.0 Alpha 4
-----------

*January 13, 2018*

1.0 Alpha 3
-----------

*January 9, 2018*

1.0 Alpha 2
-----------

*Dec 23, 2017*

1.0 Alpha 1
-----------

*Nov 24, 2017*


.. [#] Nothing wrong with @dw's excellent Python LMDB library; however,
   Lakesuperior performs heavy manipulation on data retrieved from the store which
   is more efficiently done in C/Cython.