matthieu
/
dune
1
0
Fork 0
Code Issues Pull Requests Releases Wiki Activity

Convert a few files to markdown 

Markdown works better with github
This commit is contained in:
Jeremie Dimino 2017-06-02 13:47:07 +01:00
parent fd76d7b9a9
commit b546af25c4
5 changed files with 311 additions and 293 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

0
LICENSE.txt → LICENSE.md
View File

211
README.org → README.md
View File

@ -1,4 +1,5 @@
* JBUILDER - A composable build system for OCaml and Reason
Jbuilder - A composable build system
====================================
Jbuilder is a build system designed for OCaml/Reason projects only. It
focuses on providing the user with a consistent experience and takes
@ -11,34 +12,50 @@ Street and adapted to the open source world. It has matured over a
long time and is used daily by hundred of developpers, which means
that it is highly tested and productive.
Jbuilder comes with a [[http://jbuilder.readthedocs.io/en/latest/][manual]]. If you want to get started without
reading too much, you can look at the [[http://jbuilder.readthedocs.io/en/latest/quick-start.html][quick start guide]].
Jbuilder comes with a [manual][manual]. If you want to get started
without reading too much, you can look at the
[quick start guide][quick-start].
The [[example]] directory contains examples of projects using jbuilder.
The [example]() directory contains examples of projects using
jbuilder.
[[https://travis-ci.org/janestreet/jbuilder][https://travis-ci.org/janestreet/jbuilder.png?branch=master]]
#+html: <a href="https://ci.appveyor.com/project/diml/jbuilder/branch/master"><img src="https://ci.appveyor.com/api/projects/status/bn3kcxx648jt6dyt?svg=true"/></a>
[![Travis status][travis-img]][travis] [![AppVeyor status][appveyor-img]][appveyor]
** Overview
[manual]: http://jbuilder.readthedocs.io/en/latest/
[quick-start]: http://jbuilder.readthedocs.io/en/latest/quick-start.html
[travis]: https://travis-ci.org/janestreet/jbuilder
[travis-img]: https://travis-ci.org/janestreet/jbuilder.png?branch=master
[appveyor]: https://ci.appveyor.com/project/diml/jbuilder/branch/master
[appveyor-img]: https://ci.appveyor.com/api/projects/status/bn3kcxx648jt6dyt?svg=true
[merlin]: https://github.com/ocaml/merlin
[opam]: https://opam.ocaml.org
[jenga]: https://github.com/janestreet/jenga
[issues]: https://github.com/janestreet/jbuilder/issues
[topkg-jbuilder]: https://github.com/diml/topkg-jbuilder
Jbuilder reads project metadata from =jbuild= files, which are either
Overview
--------
Jbuilder reads project metadata from `jbuild` files, which are either
static files in a simple S-expression syntax or OCaml scripts. It uses
this information to setup build rules, generate configuration files
for development tools such as [[https://github.com/ocaml/merlin][merlin]], handle installation, etc...
for development tools such as [merlin][merlin], handle installation,
etc...
Jbuilder itself is fast, has very low overhead and supports parallel
builds on all platforms. It has no system dependencies: all you need
to build jbuilder and packages using jbuilder is OCaml. You don't need
=make= or =bash= as long as the packages themselves don't use =bash=
`make` or `bash` as long as the packages themselves don't use `bash`
explicitly.
Especially, one can install OCaml on Windows with a binary installer
and then use only the Windows Console to build Jbuilder and packages
using Jbuilder.
** Strengths
Strengths
---------
*** Composable
### Composable
Take n repositories that use Jbuilder, arrange them in any way on the
file system and the result is still a single repository that Jbuilder
@ -46,20 +63,20 @@ knows how to build at once.
This make simultaneous development on multiple packages trivial.
*** Gracefully handles multi-package repositories
### Gracefully handles multi-package repositories
Jbuilder knows how to handle repositories containing several
packages. When building via [[https://opam.ocaml.org/][opam]], it is able to correctly use
packages. When building via [opam][opam], it is able to correctly use
libraries that were previously installed even if they are already
present in the source tree.
The magic invocation is:
#+begin_src sh
```sh
$ jbuilder build --only-packages <package-name> @install
#+end_src
```
*** Building against several configurations at once
### Building against several configurations at once
Jbuilder is able to build a given source code repository against
several configurations simultaneously. This helps maintaining packages
@ -67,172 +84,183 @@ across several versions of OCaml as you can tests them all at once
without hassle.
This feature should make cross-compilation easy, see details in the
[[ROADMAP.org][roadmap]].
[roadmap](ROADMAP.org).
This feature requires [[https://opam.ocaml.org/][opam]].
This feature requires [opam][opam].
*** Jenga bridge
### Jenga bridge
[[https://github.com/janestreet/jenga][Jenga]] is another build system for OCaml that has more advanced
features such as polling or much better editor integration. Jenga is
more powerful and more complex and as a result as much more
dependencies. It is planned to implement a small bridge between the
two so that a Jbuilder project can build with Jenga using this bridge.
[Jenga][jenga] is another build system for OCaml that has more
advanced features such as polling or much better editor
integration. Jenga is more powerful and more complex and as a result
as much more dependencies. It is planned to implement a small bridge
between the two so that a Jbuilder project can build with Jenga using
this bridge.
** Requirements
Requirements
------------
Jbuilder requires OCaml version 4.02.3 or greater.
** Installation
nstallation
-----------
The recommended way to install jbuilder is via the [[https://opam.ocaml.org][opam package manager]]:
The recommended way to install jbuilder is via the
[opam package manager][opam]:
#+begin_src sh
```sh
$ opam install jbuilder
#+end_src
```
You can also build it manually with:
#+begin_src sh
```sh
$ make
$ make install
#+end_src
```
Note however that =make install= requires the =opam-installer= tool.
Note however that `make install` requires the `opam-installer` tool.
If you do not have =make=, you can do the following:
If you do not have `make`, you can do the following:
#+begin_src sh
```sh
$ ocaml bootstrap.ml
$ ./boot.exe
$ ./_build/default/bin/main.exe install
#+end_src
```
** Support
Support
-------
If you have questions about jbuilder, you can send an email to
ocaml-core@googlegroups.com or [[https://github.com/janestreet/jbuilder/issues][open a ticket on github]].
ocaml-core@googlegroups.com or [open a ticket on github][issues].
** Status
Status
------
Jbuilder is now in beta testing stage. Once a bit more testing has
been done, it will be released in 1.0.
** Roadmap
Roadmap
-------
See [[ROADMAP.org]] for the current plan. Help on any of these points is
welcome!
See [the roadmap](ROADMAP.org) for the current plan. Help on any of
these points is welcome!
** FAQ
FAQ
---
*** Why do many Jbuilder projects contain a Makefile?
### Why do many Jbuilder projects contain a Makefile?
Many Jbuilder project contain a toplevel =Makefile=. It is often only
Many Jbuilder project contain a toplevel `Makefile`. It is often only
there only for convenience, for the following reasons:
1. there are many different build systems out there, all with a
different CLI. If you have been hacking for a long time, the one
true invocation you know is =make && make install=, possibly
preceded by =./configure=
true invocation you know is `make && make install`, possibly
preceded by `./configure`
2. you often have a few common operations that are not part of the
build and =make <blah>= is a good way to provide them
build and `make <blah>` is a good way to provide them
3. =make= is shorter to type than =jbuilder build @install=
3. `make` is shorter to type than `jbuilder build @install`
*** How to add a configure step to a jbuilder project?
### How to add a configure step to a jbuilder project?
[[example/sample-projects/with-configure-step]] shows one way to do it
[example/sample-projects/with-configure-step]() shows one way to do it
that preserves composability; i.e. it doesn't require to manually run
all =./configure= script when working on multiple projects at the same
all `./configure` script when working on multiple projects at the same
time.
*** Can I use topkg with jbuilder?
### Can I use topkg with jbuilder?
Yes, have a look at the [[https://github.com/diml/topkg-jbuilder][topkg-jbuilder]] project for more details.
Yes, have a look at the [topkg-jbuilder][topkg-jbuilder] project for
more details.
** Known issues
Known issues
------------
*** Optional libraries inside a multilib directory
### Optional libraries inside a multilib directory
[[https://github.com/janestreet/jbuilder/issues/51]]
https://github.com/janestreet/jbuilder/issues/51
If a directory contains several libraries and some are marked as
optional (by adding =(optional)= in the =(library ...)= stanza), then
optional (by adding `(optional)` in the `(library ...)` stanza), then
the dependencies will still be required to perform the build.
This could be sorted out with some refactoring, but there is a simple
workaround, so it is low-priority.
**** Workaround
#### Workaround
Put each optional library in a separate directory.
*** mli only modules
### mli only modules
https://github.com/janestreet/jbuilder/issues/9
Due to the low-level details of OCaml compilation, it is currently
possible to write a module that has only a =.mli= and no =.ml=
possible to write a module that has only a `.mli` and no `.ml`
file. This works as long as the mli contains only type declarations.
This is not a properly supported feature of the compiler, and in
particular it is not possible to alias such modules or use them as the
argument of a functor. Moreover, if you do write a value declaration,
or even just define an exception in the =.mli=, then you won't get an
or even just define an exception in the `.mli`, then you won't get an
error until the point where you link an executable using this module.
For these reason, mli only modules are not recommended by Jbuilder
until the compiler support them properly.
**** Workaround
#### Workaround
As long as a module type contains no value declaration, it is possible
to turn in to an implementation by using a recursive module:
#+begin_src ocaml
```ocaml
module rec M : sig
type t = A | B
end = M
include M
#+end_Src
```
So if you have a module without a =.ml= file, simply generate a =.ml=
from the =.mli= using this trick. For instance you can add the
So if you have a module without a `.ml` file, simply generate a `.ml`
from the `.mli` using this trick. For instance you can add the
following rule into your jbuild file:
#+begin_src scheme
```scheme
(rule (with-output-to foo.ml
(progn
(echo "module rec HACK : sig\n")
(cat foo.mli)
(echo "\nend = HACK\ninclue HACK\n"))))
#+end_src
```
In fact, jbuilder will automatically add this rule if you have a
module without imlpementation. However it will print a warning.
** Implementation details
Implementation details
----------------------
This section is for people who want to work on Jbuilder itself.
*** Bootstrap
### Bootstrap
In order to build itself, Jbuilder uses an OCaml script ([[bootstrap.ml]])
that dumps most of the sources of Jbuilder into a single =boot.ml=
file. This file is built using =ocamlopt= or =ocamlc= and used to
build everything else.
In order to build itself, Jbuilder uses an OCaml script
([bootstrap.ml]()) that dumps most of the sources of Jbuilder into a
single =boot.ml= file. This file is built using =ocamlopt= or =ocamlc=
and used to build everything else.
*** OCaml compatibility test
### OCaml compatibility test
Install opam switches for all the entries in the [[jbuild-workspace.dev]]
file and run:
Install opam switches for all the entries in the
[jbuild-workspace.dev]() file and run:
#+begin_src sh
```sh
$ make all-supported-ocaml-versions
#+end_src
```
*** Repository organization
### Repository organization
- =vendor/= contains dependencies of Jbuilder, that have been vendored
- =plugin/= contains the API given to =jbuild= files that are OCaml
@ -242,7 +270,7 @@ $ make all-supported-ocaml-versions
- =bin/= contains the command line interface
- =doc/= contains the manual and rules to generate the manual pages
*** Design
### Design
Jbuilder was initially designed to sort out the public release of Jane
Street packages which became incredibly complicated over time. It is
@ -255,18 +283,19 @@ tree. This is used to automatically generate the =<package>.opam=
files for all Jane Street packages.
To implement this, the build rules are described using a build arrow,
which is defined in [[src/build.mli][src/build]]. In the end it makes the development
of the internal rules of Jbuilder very composable and quite pleasant.
which is defined in [src/build.mli](). In the end it makes the
development of the internal rules of Jbuilder very composable and
quite pleasant.
To deal with process multiplexing, Jbuilder uses a simplified
Lwt/Async-like monad, implemented in [[src/future.mli][src/future]].
Lwt/Async-like monad, implemented in [src/future.mli]().
**** Code flow
#### Code flow
- [[src/jbuild_types.ml][src/jbuild_types]] contains the internal representation of =jbuild=
- [src/jbuild_types.ml]() contains the internal representation of =jbuild=
files and the parsing code
- [[src/jbuild_load.ml][src/jbuild_load]] contains the code to scan a source tree and build
the internal database by reading the =jbuild= files
- [[src/gen_rules.ml][src/gen_rules]] contains all the build rules of Jbuilder
- [[src/build_system.ml][src/build_system]] contains a trivial implementation of a Build
- [src/jbuild_load.ml]() contains the code to scan a source tree and
build the internal database by reading the =jbuild= files
- [src/gen_rules.ml]() contains all the build rules of Jbuilder
- [src/build_system.ml]() contains a trivial implementation of a Build
system. This is what Jenga will provide when implementing the bridge

190
ROADMAP.md Normal file
View File

@ -0,0 +1,190 @@
This document describe the plan for the next releases of Jbuilder.
# V1
Jbuilder is currently being tested for various packages. Once we are
confident there is enough for most OCaml packages, the version 1.0
will be released and the version 1 of the jbuild specification will be
frozen. Additions to the spec will go into version 1.1 of the spec and
breaking changes will go into version 2.
However, until version 1.0 of Jbuilder is released, breaking changes
to the jbuild specification are not accepted in order to allow
packages depending on jbuilder to be released.
## ~~CLI~~
Add a proper [cmdliner](http://erratique.ch/software/cmdliner) based
CLI. Jbuilder will include a copy of cmdliner to avoid the extra
dependency.
- *20/02/2017*: This is now implemented thanks to Rudi Grinberg (#5)
### Improve the man pages
Add a bit more documentation in the Man pages generated by cmdliner.
## ~~Documentation~~
Document the usage and design of Jbuilder.
- *21/02/2017*: There is now a manual, it still needs a bit more about
CLI usage
- *07/03/2017*: Manual is complete
## ~~Stable jbuild types~~
Add a stable version of the jbuild format so that one can write
`(jbuild_version 1)` inside jbuild files and be sure that they will
work with future versions of jbuild.
- *24/02/2017*: implemented
## ~~Finding the project/workspace root~~
Currently `jbuilder` assumes that the root of the project/workspace is
where it is started. Eventually this will be changed as follows:
- if there is a `jbuild-workspace` in a parent directory, it marks the root;
- if not found, look for a `opam` or `package.opam` file in parent directories;
- if not found, look for a `.git`, `.hg`, ... file in parent directories;
- if not found, use the current directory as root.
- *28/02/2017*: Implemented
## ~~Generate .merlin files #1~~
- *25/02/2017*: Implemented by Richard Davison (#2)
## ~~Running tests with jbuilder #3~~
Allow to define tests with `(alias ...)` stanzas and add `jbuilder
runtest` to run them.
- *21/02/2017*: Rudi Grinberg added support for aliases (#7)
- *23/02/2017*: Added a `runtest` subcommand
## ~~Implement package version support~~
Implement finding the version of a package, as described in the
documentation.
- *24/02/2017*: Implemented
## ~~Add a `(package ...)` field to alias stanzas~~
So that we can filter tests when using `-p pkg`
- *05/02/2017*: Rudi Grinberg added this ([[https://github.com/janestreet/jbuilder/pull/64][#64]])
## Integrate rules for `-output-complete-obj`
#23
## ~~Integrate rules for js_of_ocaml~~
#60
- *02/05/2017*: Merged
## ~~Topkg integration~~
It would be nice to integrate with
[topkg](http://erratique.ch/software/topkg). Some of the features
overlap, and for instance there doesn't seem to be much point in using
topkg to generate the .install or invoke jbuilder since jbuilder
already handles that well. However, the other features available
through the `topkg` command line tool would be good to have.
- *08/05/2017*: Made topkg and jbuilder work together with
[topkg-jbuilder](https://github.com/diml/topkg-jbuilder)
## ~~odoc support~~
Support generating documentation with
[odoc](https://github.com/ocaml-doc/odoc)
# After V1
## Cross-compilation
Everything needed for cross-compilation is implemented. One
essentially need to add a function `host_exe : Path.t -> Path.t`
inside build contexts to make it all work, as well as a way to define
the build contexts. These could be defined inside `jbuild-workspace`
as follows:
```scheme
(context
((name foo)
(switch 4.04.0)))
(context
((name foo+mingw)
(switch 4.04.0+mingw)
(host foo)))
```
## Jenga bridge
Implement a jenga plugin that can read the same jbuild files as
Jbuilder. To do that we'll use Jbuilder as a library.
## Inline tests
Setup automatic support of
[inline tests](https://github.com/janestreet/ppx_inline_test) and
[inline benchmarks](https://github.com/janestreet/ppx_bench).
## Extend the action language
Currently in `(action ...)` fields, when not using `bash` the language
is very limited. It would be nice to add more commands that would
guarantee portability and avoid the quoting nightmare of `bash`.
FS commands should be straight foward to implement:
- `(copy <src> <dst>)`
- `(mkdir <path>)`
- ...
Redirections to/from files are simple as well.
We could also implements pipes (`(pipe <command1> <command2> ...)`) by
using temporary files. Using proper pipes would complicate windows
support and would make proper handling of `-j` hard. Using temporary
files will be just fine.
## User configuration file
Load a configuration file from `~/.config/jbuilder/config.sexp` where
the user can define preferences such as colors.
## Code improvements
### Delete the global variables in Clflags
### Consolidate the S-expression parser
It doesn't follow the specification given in the readme of
[parsexp](https://github.com/janestreet/parsexp). This need to be
fixed.
## Make Jbuild_plugin a library
Currently Jbuilder generates a wrapper script containing the source
code of the `Jbuild_plugin` followed by the user script. While this
method is trivial to implement, it is not great if users want to write
libraries for jbuild plugins.
What we should do instead is create a proper `jbuild_plugin` library
that is installed. This library should read a file containing the
build context details generated by Jbuilder and passed as
`Sys.argv.(1)`.
We need to refactor things a bit to make this happen, in particular
the library will propably need to know how to parse s-expression. We
can create a `jbuild_common` library to put the parts that are common
between `jbuild_plugin` and `jbuilder`.
Note that `doc/jbuild` is an OCaml script. To simplify the bootstrap,
we should just convert it back to a static `jbuild` file.

194
ROADMAP.org
View File

@ -1,194 +0,0 @@
This document describe the plan for the next releases of Jbuilder.
* V1
Jbuilder is currently being tested for various packages. Once we are
confident there is enough for most OCaml packages, the version 1.0
will be released and the version 1 of the jbuild specification will be
frozen. Additions to the spec will go into version 1.1 of the spec and
breaking changes will go into version 2.
However, until version 1.0 of Jbuilder is released, breaking changes
to the jbuild specification are not accepted in order to allow
packages depending on jbuilder to be released.
** +CLI+
Add a proper [[http://erratique.ch/software/cmdliner][cmdliner]] based CLI.
Jbuilder will include a copy of cmdliner to avoid the extra dependency.
- *20/02/2017*: This is now implemented thanks to Rudi Grinberg ([[https://github.com/janestreet/jbuilder/pull/2][#5]])
*** Improve the man pages
Add a bit more documentation in the Man pages generated by cmdliner.
** +Documentation+
Document the usage and design of Jbuilder.
- *21/02/2017*: There is now a manual, it still needs a bit more about
CLI usage
- *07/03/2017*: Manual is complete
** +Stable jbuild types+
Add a stable version of the jbuild format so that one can write
=(jbuild_version 1)= inside jbuild files and be sure that they will
work with future versions of jbuild.
- *24/02/2017*: implemented
** +Finding the project/workspace root+
Currently =jbuilder= assumes that the root of the project/workspace is
where it is started. Eventually this will be changed as follows:
- if there is a =jbuild-workspace= in a parent directory, it marks the root;
- if not found, look for a =opam= or =package.opam= file in parent directories;
- if not found, look for a =.git=, =.hg=, ... file in parent directories;
- if not found, use the current directory as root.
- *28/02/2017*: Implemented
** +Generate .merlin files [[https://github.com/janestreet/jbuilder/issues/1][#1]]+
- *25/02/2017*: Implemented by Richard Davison ([[https://github.com/janestreet/jbuilder/pull/2][#2]])
** +Running tests with jbuilder [[https://github.com/janestreet/jbuilder/issues/3][#3]]+
Allow to define tests with =(alias ...)= stanzas and add =jbuilder
runtest= to run them.
- *21/02/2017*: Rudi Grinberg added support for aliases ([[https://github.com/janestreet/jbuilder/pull/2][#7]])
- *23/02/2017*: Added a =runtest= subcommand
** +Implement package version support+
Implement finding the version of a package, as described in the
documentation.
- *24/02/2017*: Implemented
** +Add a =(package ...)= field to alias stanzas+
So that we can filter tests when using =-p pkg=
- *05/02/2017*: Rudi Grinberg added this ([[https://github.com/janestreet/jbuilder/pull/64][#64]])
** Integrate rules for =-output-complete-obj=
https://github.com/janestreet/jbuilder/pull/23
** +Integrate rules for js_of_ocaml+
https://github.com/janestreet/jbuilder/pull/60
- *02/05/2017*: Merged
** +Topkg integration+
It would be nice to integrate with [[http://erratique.ch/software/topkg][topkg]]. Some of the features
overlap, and for instance there doesn't seem to be much point in using
topkg to generate the .install or invoke jbuilder since jbuilder
already handles that well. However, the other features available
through the =topkg= command line tool would be good to have.
- *08/05/2017*: Made topkg and jbuilder work together with [[https://github.com/diml/topkg-jbuilder][topkg-jbuilder]]
** odoc support
Support generating documentation with [[https://github.com/ocaml-doc/odoc][odoc]].
* After V1
** Cross-compilation
Everything needed for cross-compilation is implemented. One
essentially need to add a function =host_exe : Path.t -> Path.t=
inside build contexts to make it all work, as well as a way to define
the build contexts. These could be defined inside =jbuild-workspace=
as follows:
#+begin_src scheme
(context
((name foo)
(switch 4.04.0)))
(context
((name foo+mingw)
(switch 4.04.0+mingw)
(host foo)))
#+end_src
** Jenga bridge
Implement a jenga plugin that can read the same jbuild files as
Jbuilder. To do that we'll use Jbuilder as a library.
** Inline tests
Setup automatic support of [[https://github.com/janestreet/ppx_inline_test][inline tests]] and [[https://github.com/janestreet/ppx_bench][inline benchmarks]].
** Extend the action language
Currently in =(action ...)= fields, when not using =bash= the language
is very limited. It would be nice to add more commands that would
guarantee portability and avoid the quoting nightmare of =bash=.
FS commands should be straight foward to implement:
- =(copy <src> <dst>)=
- =(mkdir <path>)=
- ...
Redirections to/from files are simple as well.
We could also implements pipes (=(pipe <command1> <command2> ...)=) by
using temporary files. Using proper pipes would complicate windows
support and would make proper handling of =-j= hard. Using temporary
files will be just fine.
** User configuration file
Load a configuration file from =~/.config/jbuilder/config.sexp= where
the user can define preferences such as colors.
** Code improvements
*** Delete the global variables in Clflags
*** Improve the Action.Unexpanded.t
=String_with_vars.t= should take a type parameter and we would have:
#+begin_src
type variable =
| Plain of string
| Bin of string
| Lib of string * string
| ...
type user_action = variable Action.Unexpanded.t
#+end_src
This would allow to report parsing errors immediately and at the right
location.
*** Consolidate the S-expression parser
It doesn't follow the specification given in the readme of
[[https://github.com/janestreet/parsexp][parsexp]]. This need to be fixed.
** Make =Jbuild_plugin= a library
Currently Jbuilder generates a wrapper script containing the source
code of the =Jbuild_plugin= followed by the user script. While this
method is trivial to implement, it is not great if users want to write
libraries for jbuild plugins.
What we should do instead is create a proper =jbuild_plugin= library
that is installed. This library should read a file containing the
build context details generated by Jbuilder and passed as
=Sys.argv.(1)=.
We need to refactor things a bit to make this happen, in particular
the library will propably need to know how to parse s-expression. We
can create a =jbuild_common= library to put the parts that are common
between =jbuild_plugin= and =jbuilder=.
Note that =doc/jbuild= is an OCaml script. To simplify the bootstrap,
we should just convert it back to a static =jbuild= file.

9
pkg/pkg.ml
View File

@ -1,9 +1,2 @@
#use "topfind"
#require "topkg-jbuilder"
open Topkg
let () =
Topkg_jbuilder.describe ()
~readmes: [ Pkg.std_file "README.org" ]
~licenses:[ Pkg.std_file "LICENSE.txt" ]
#require "topkg-jbuilder.auto"