diff --git a/HACKING.md b/HACKING.md new file mode 100644 index 00000000..a83b3618 --- /dev/null +++ b/HACKING.md @@ -0,0 +1,69 @@ +# Hacking on Dune + +This section is for people who want to work on Jbuilder itself. + +## Bootstrap + +In order to build itself, Jbuilder uses an OCaml script +([bootstrap.ml](bootstrap.ml)) that dumps most of the sources of Jbuilder into a +single `boot.ml` file. This file is built using `ocamlopt` or `ocamlc` +and used to build everything else. + +Note that we don't include all of the sources in boot.ml. We skip a +few parts to speed up the build. In particular: +- vendored libraries are replaced by simpler implementations taken + from `vendor/boot` +- a few files in `src` have an alternative version. These alternatives + versions are named `XXX.boot.EXT`. For instance: `glob_lexer.boot.ml` + +## OCaml compatibility test + +Install opam switches for all the entries in the +[jbuild-workspace.dev](jbuild-workspace.dev) file and run: + +```sh +$ make all-supported-ocaml-versions +``` + +## Repository organization + +- `vendor/` contains dependencies of Jbuilder, that have been vendored +- `plugin/` contains the API given to `jbuild` files that are OCaml + scripts +- `src/` contains the core of `Jbuilder`, as a library so that it can + be used to implement the Jenga bridge later +- `bin/` contains the command line interface +- `doc/` contains the manual and rules to generate the manual pages + +## Design + +Jbuilder was initially designed to sort out the public release of Jane +Street packages which became incredibly complicated over time. It is +still successfully used for this purpose. + +One necessary feature to achieve this is the ability to precisely +report the external dependencies necessary to build a given set of +targets without running any command, just by looking at the source +tree. This is used to automatically generate the `.opam` +files for all Jane Street packages. + +To implement this, the build rules are described using a build arrow, +which is defined in [src/build.mli](src/build.mli). In the end it makes the +development of the internal rules of Jbuilder very composable and +quite pleasant. + +To deal with process multiplexing, Jbuilder uses a simplified +Lwt/Async-like monad, implemented in [src/future.mli](src/future.mli). + +## Code flow + +- [src/jbuild.mli](src/jbuild.mli) contains the internal representation + of `jbuild` files and the parsing code +- [src/jbuild_load.mli](src/jbuild_load.mli) contains the code to scan + a source tree and build the internal database by reading + the `jbuild` files +- [src/gen_rules.mli](src/gen_rules.mli) contains all the build rules + of Jbuilder +- [src/build_system.mli](src/build_system.mli) contains a trivial + implementation of a Build system. This is what Jenga will provide + when implementing the bridge diff --git a/README.md b/README.md index a66a5659..6ffa257b 100644 --- a/README.md +++ b/README.md @@ -145,133 +145,9 @@ ocaml-core@googlegroups.com or [open a ticket on github][issues]. Status ------ -Jbuilder is now in beta testing stage. Once a bit more testing has -been done, it will be released in 1.0. +Dune is now fairly stable and is used by the majority of packages on opam. +Nevertheless, breaking changes are expected until 1.0. The release of 1.0 will +also include the jbuilder to dune renaming. This renaming will do away with the +old name entirely. While this will be a breaking change, dune will provide a +tool to automatically migrate pre 1.0 projects. -Roadmap -------- - -See [the roadmap](ROADMAP.md) for the current plan. Help on any of -these points is welcome! - -FAQ ---- - -### Why do many Jbuilder projects contain a Makefile? - -Many Jbuilder project contain a toplevel `Makefile`. It is often only -there for convenience, for the following reasons: - -1. there are many different build systems out there, all with a - different CLI. If you have been hacking for a long time, the one - true invocation you know is `make && make install`, possibly - preceded by `./configure` - -2. you often have a few common operations that are not part of the - build and `make ` is a good way to provide them - -3. `make` is shorter to type than `jbuilder build @install` - -### How to add a configure step to a jbuilder project? - -[example/sample-projects/with-configure-step](example/sample-projects/with-configure-step) shows -one way to do it which preserves composability; i.e. it doesn't require manually -running `./configure` script when working on multiple projects at the same time. - -### Can I use topkg with jbuilder? - -Yes, have a look at the [topkg-jbuilder][topkg-jbuilder] project for -more details. - -### Where can I find some examples of projects using Jbuilder? - -The [dune-universe](https://github.com/dune-universe/dune-universe) -repository contains a snapshot of the latest versions of all opam packages -depending on jbuilder. It is therefore a useful reference to search through -to find different approaches to constructing build rules. - -Known issues ------------- - -### mli only modules - -These are supported, however using them might cause make it impossible -for non-jbuilder users to use your library. We tried to use them for -some internal module generated by Jbuilder and it broke the build of -projects not using Jbuilder: - -https://github.com/ocaml/dune/issues/567 - -So, while they are supported, you should be careful where you use -them. Using a `.ml` only module is still preferable. - -Implementation details ----------------------- - -This section is for people who want to work on Jbuilder itself. - -### Bootstrap - -In order to build itself, Jbuilder uses an OCaml script -([bootstrap.ml](bootstrap.ml)) that dumps most of the sources of Jbuilder into a -single `boot.ml` file. This file is built using `ocamlopt` or `ocamlc` -and used to build everything else. - -Note that we don't include all of the sources in boot.ml. We skip a -few parts to speed up the build. In particular: -- vendored libraries are replaced by simpler implementations taken - from `vendor/boot` -- a few files in `src` have an alternative version. These alternatives - versions are named `XXX.boot.EXT`. For instance: `glob_lexer.boot.ml` - -### OCaml compatibility test - -Install opam switches for all the entries in the -[jbuild-workspace.dev](jbuild-workspace.dev) file and run: - -```sh -$ make all-supported-ocaml-versions -``` - -### Repository organization - -- `vendor/` contains dependencies of Jbuilder, that have been vendored -- `plugin/` contains the API given to `jbuild` files that are OCaml - scripts -- `src/` contains the core of `Jbuilder`, as a library so that it can - be used to implement the Jenga bridge later -- `bin/` contains the command line interface -- `doc/` contains the manual and rules to generate the manual pages - -### Design - -Jbuilder was initially designed to sort out the public release of Jane -Street packages which became incredibly complicated over time. It is -still successfully used for this purpose. - -One necessary feature to achieve this is the ability to precisely -report the external dependencies necessary to build a given set of -targets without running any command, just by looking at the source -tree. This is used to automatically generate the `.opam` -files for all Jane Street packages. - -To implement this, the build rules are described using a build arrow, -which is defined in [src/build.mli](src/build.mli). In the end it makes the -development of the internal rules of Jbuilder very composable and -quite pleasant. - -To deal with process multiplexing, Jbuilder uses a simplified -Lwt/Async-like monad, implemented in [src/future.mli](src/future.mli). - -#### Code flow - -- [src/jbuild.mli](src/jbuild.mli) contains the internal representation - of `jbuild` files and the parsing code -- [src/jbuild_load.mli](src/jbuild_load.mli) contains the code to scan - a source tree and build the internal database by reading - the `jbuild` files -- [src/gen_rules.mli](src/gen_rules.mli) contains all the build rules - of Jbuilder -- [src/build_system.mli](src/build_system.mli) contains a trivial - implementation of a Build system. This is what Jenga will provide - when implementing the bridge diff --git a/ROADMAP.md b/ROADMAP.md deleted file mode 100644 index f1a678ff..00000000 --- a/ROADMAP.md +++ /dev/null @@ -1,162 +0,0 @@ -This document describe the plan for the next releases of Jbuilder. - -# V1 - -Jbuilder is currently in feature freeze. V1 should be released soon. - -## ~~CLI~~ - -Add a proper [cmdliner](http://erratique.ch/software/cmdliner) based -CLI. Jbuilder will include a copy of cmdliner to avoid the extra -dependency. - -- *20/02/2017*: This is now implemented thanks to Rudi Grinberg (#5) - -### Improve the man pages - -Add a bit more documentation in the Man pages generated by cmdliner. - -## ~~Documentation~~ - -Document the usage and design of Jbuilder. - -- *21/02/2017*: There is now a manual, it still needs a bit more about - CLI usage -- *07/03/2017*: Manual is complete - -## ~~Stable jbuild types~~ - -Add a stable version of the jbuild format so that one can write -`(jbuild_version 1)` inside jbuild files and be sure that they will -work with future versions of jbuild. - -- *24/02/2017*: implemented - -## ~~Finding the project/workspace root~~ - -Currently `jbuilder` assumes that the root of the project/workspace is -where it is started. Eventually this will be changed as follows: - -- if there is a `jbuild-workspace` in a parent directory, it marks the root; -- if not found, look for a `opam` or `package.opam` file in parent directories; -- if not found, look for a `.git`, `.hg`, ... file in parent directories; -- if not found, use the current directory as root. - -- *28/02/2017*: Implemented - -## ~~Generate .merlin files #1~~ - -- *25/02/2017*: Implemented by Richard Davison (#2) - -## ~~Running tests with jbuilder #3~~ - -Allow to define tests with `(alias ...)` stanzas and add `jbuilder -runtest` to run them. - -- *21/02/2017*: Rudi Grinberg added support for aliases (#7) -- *23/02/2017*: Added a `runtest` subcommand - -## ~~Implement package version support~~ - -Implement finding the version of a package, as described in the -documentation. - -- *24/02/2017*: Implemented - -## ~~Add a `(package ...)` field to alias stanzas~~ - -So that we can filter tests when using `-p pkg` - -- *05/02/2017*: Rudi Grinberg added this ([[https://github.com/ocaml/dune/pull/64][#64]]) - -## ~~Integrate rules for js_of_ocaml~~ - -#60 - -- *02/05/2017*: Merged - -## ~~Topkg integration~~ - -It would be nice to integrate with -[topkg](http://erratique.ch/software/topkg). Some of the features -overlap, and for instance there doesn't seem to be much point in using -topkg to generate the .install or invoke jbuilder since jbuilder -already handles that well. However, the other features available -through the `topkg` command line tool would be good to have. - -- *08/05/2017*: Made topkg and jbuilder work together with - [topkg-jbuilder](https://github.com/diml/topkg-jbuilder) - -## ~~odoc support~~ - -Support generating documentation with -[odoc](https://github.com/ocaml-doc/odoc) - -# After V1 - -## Integrate rules for `-output-complete-obj` - -#23 - -## Cross-compilation - -Everything needed for cross-compilation is implemented. One -essentially need to add a function `host_exe : Path.t -> Path.t` -inside build contexts to make it all work, as well as a way to define -the build contexts. These could be defined inside `jbuild-workspace` -as follows: - -```scheme -(context - ((name foo) - (switch 4.04.0))) - -(context - ((name foo+mingw) - (switch 4.04.0+mingw) - (host foo))) -``` - -## Jenga bridge - -Implement a jenga plugin that can read the same jbuild files as -Jbuilder. To do that we'll use Jbuilder as a library. - -## Inline tests - -Setup automatic support of -[inline tests](https://github.com/janestreet/ppx_inline_test) and -[inline benchmarks](https://github.com/janestreet/ppx_bench). - -## Extend the action language - -Currently in `(action ...)` fields, when not using `bash` the language -is very limited. It would be nice to add more commands that would -guarantee portability and avoid the quoting nightmare of `bash`. - -FS commands should be straight foward to implement: -- `(copy )` -- `(mkdir )` -- ... - -Redirections to/from files are simple as well. - -We could also implements pipes (`(pipe ...)`) by -using temporary files. Using proper pipes would complicate windows -support and would make proper handling of `-j` hard. Using temporary -files will be just fine. - -## User configuration file - -Load a configuration file from `~/.config/jbuilder/config.sexp` where -the user can define preferences such as colors. - -## Code improvements - -### Delete the global variables in Clflags - -### Consolidate the S-expression parser - -It doesn't follow the specification given in the readme of -[parsexp](https://github.com/janestreet/parsexp). This need to be -fixed. diff --git a/doc/faq.rst b/doc/faq.rst new file mode 100644 index 00000000..13e1c007 --- /dev/null +++ b/doc/faq.rst @@ -0,0 +1,40 @@ +*** +FAQ +*** + +Why do many Jbuilder projects contain a Makefile? +================================================= + +Many Jbuilder project contain a toplevel `Makefile`. It is often only there for +convenience, for the following reasons: + +1. there are many different build systems out there, all with a different CLI. + If you have been hacking for a long time, the one true invocation you know is + `make && make install`, possibly preceded by `./configure` + +2. you often have a few common operations that are not part of the build and + `make ` is a good way to provide them + +3. `make` is shorter to type than `jbuilder build @install` + +How to add a configure step to a jbuilder project? +================================================== + +[example/sample-projects/with-configure-step](example/sample-projects/with-configure-step) +shows one way to do it which preserves composability; i.e. it doesn't require +manually running `./configure` script when working on multiple projects at the +same time. + +Can I use topkg with jbuilder? +============================== + +Yes, have a look at the [topkg-jbuilder][topkg-jbuilder] project for +more details. + +here can I find some examples of projects using Jbuilder? +========================================================= + +The [dune-universe](https://github.com/dune-universe/dune-universe) repository +contains a snapshot of the latest versions of all opam packages depending on +jbuilder. It is therefore a useful reference to search through to find different +approaches to constructing build rules. diff --git a/doc/index.rst b/doc/index.rst index 32e11241..b8bb2f1a 100644 --- a/doc/index.rst +++ b/doc/index.rst @@ -18,3 +18,5 @@ Welcome to jbuilder's documentation! documentation usage advanced-topics + faq + known-issues diff --git a/doc/known-issues.rst b/doc/known-issues.rst new file mode 100644 index 00000000..d02a9655 --- /dev/null +++ b/doc/known-issues.rst @@ -0,0 +1,17 @@ +************ +Known Issues +************ + +mli only modules +================ + +These are supported, however using them might cause make it impossible for +non-jbuilder users to use your library. We tried to use them for some internal +module generated by Jbuilder and it broke the build of projects not using +Jbuilder: + +https://github.com/ocaml/dune/issues/567 + +So, while they are supported, you should be careful where you use them. Using a +`.ml` only module is still preferable. +