Page MenuHomePhorge
Files F117780144

D89.1775246591.diff
No OneTemporary
Actions

Authored By
Unknown
Size
17 KB
Referenced Files
None
Subscribers
None

D89.1775246591.diff
View Options

diff --git a/source/developer-guide/process.rst b/source/developer-guide/process.rst
new file mode 100644
--- /dev/null
+++ b/source/developer-guide/process.rst
@@ -0,0 +1,500 @@
+.. _developer-guide-process:
+
+===================
+Development Process
+===================
+
+.. todo::
+
+ Describe continuous integration
+
+.. todo::
+
+ Describe the difference between build, unit, functional, install,
+ integration, performance and acceptance tests.
+
+.. todo::
+
+ Describe continuous deployment
+
+.. todo::
+
+ Describe how to create a ticket
+
+.. todo::
+
+ Describe how to submit a patch
+
+.. todo::
+
+ Describe how to review and accept or decline a patch
+
+.. todo::
+
+ Describe how to formulate a feature request
+
+Kolab Groupware development largely follows an iterative and incremental
+agile software development metholodogy also known as `Scrum`_.
+
+In summary, the Kolab Groupware development process is divided into
+stages, with different teams participating in getting tasks to the next
+stage of the process.
+
+Kolab Groupware is a collaboration suite consisting of many components,
+each of them separate software development projects, some of them
+dependent on third-party software development projects, and some of them
+separate altogether. This creates some discrepancies in applying vanilla
+Scrum practices to all projects in the Kolab universe.
+
+Furthermore, Kolab Groupware is a community software development
+project, and not all participants in the community can be expected to be
+familiar with (otherwise professional) software development processes.
+
+It is therefore that we document here how we work with `Scrum`_ using
+`Phabricator`_.
+
+Benefits of Phabricator, Scrum and Test-Driven Development
+==========================================================
+
+A plethora of benefits exist in using Phabricator and Scrum over the
+current processes (which do not fall under any known nomenclature), and
+Test-Driven Development (TDD).
+
+* The processes and facilities are focused on structured software
+ development.
+
+* Software development stands in stark contrast with longer-term
+ commitments such as support.
+
+* A developer is supposed to work on tomorrow's version of the
+ software, and not yesterday's, as every distraction with yesterday's
+ software stands in the way of the focus and dedication needed to
+ provide tomorrow's innovative and competitive software.
+
+* And every distraction stands in the way of the personal and
+ professional development of the individual developer.
+
+* We aim to pull product support away from software development.
+
+* A Bugzilla is not suitable for agile software development (even with
+ tooling such as Scrumbugz), and the former point may have
+ illustrated we wish software development to be agile.
+
+* Various factors in the world of IT as well as common software
+ development practices predicate a shift toward continuous deployment
+ (at one cost) over sometimes overly precautious long-term support
+ methodologies (at another cost).
+
+* The dynamics of these costs work in favor of a level of agility in
+ software development as well as operations, especially with smaller
+ teams.
+
+Getting Acquainted with Phabricator
+===================================
+
+We use `Projects`_ in `Phabricator`_ as a means to group people and
+assign them roles, rights and responsibilities in a broader process.
+
+Tickets are created as `Maniphest`_ tasks, and roll on forward through
+the process from thereon.
+
+Software Development Projects
+-----------------------------
+
+The first type of project is a **Software Development Project**. These
+are generally associated with one or more source code management
+repositories.
+
+Each software development project is responsible for a single software
+component included in Kolab Groupware, such as **chwala** or
+**pykolab**.
+
+Because the Kolab community is an open community, membership to these
+software development projects is *open* -- meaning registered users can
+join and leave as they like.
+
+Authorization Groups (Developer Teams)
+--------------------------------------
+
+Each software development project that operates source code management
+repositories also has a group associated with it that is used for
+authorization to push to the associated source code management
+repositories.
+
+For the **PyKolab** software development project for example, a group
+**PyKolab Developers** has access to push to the ``pykolab`` GIT
+repository.
+
+These groups are mostly self-regulated, in that members of the group can
+add other members to the group.
+
+Teams and Roles
+---------------
+
+Teams and roles that are not software development teams include teams
+such as `Product Owners`_, `Scrum Masters`_ and `Quality Assurance`_.
+
+Sprints
+-------
+
+`Sprints`_ are what is in the name. They are a time-limited series of
+events resulting in deliverables being emitted at the end of the sprint.
+
+This series of events consists of:
+
+**Planning Meeting**
+
+ Held at the beginning of each sprint, the `Product Owners`_ and the
+ developer team negotiate which tasks on the Sprint's backlog they
+ will attempt to complete.
+
+ The `Product Owners`_ determine the priority of tasks and declare
+ which (of all tasks) render the most business value.
+
+ The development team determines the amount of work they feel they
+ can achieve during the sprint without incurring technical debt.
+
+ .. NOTE::
+
+ The `Product Owners`_ use tasks related to individual software
+ development projects and associate them with the **next**
+ sprint in order to build an initial sprint backlog.
+
+ Working the other way around, where the team in collaboration
+ with the `Product Owners`_ determine which tasks to put on the
+ sprint backlog doesn't scale to the number of individual
+ software development projects for Kolab Groupware as a whole.
+
+**Daily Scrums**
+
+ Short, daily stand-up meetings are used to let team members
+ summarize yesterday's work, today's work and talk about any
+ impediments encountered.
+
+ This ensures team members communicate early and often about their
+ work, problems they find, and facilitate the fast exchange of
+ knowledge and ideas or call for help.
+
+**Review Meeting**
+
+
+
+**Retrospective Meeting**
+
+Process Stages
+==============
+
+The stages of the development process largely align with a manufacturing
+process of, say, cars.
+
+.. _developer-guide-process-stage-concept:
+
+Concept
+-------
+
+This stage is an almost completely iterative process; Ideas are
+generated, possibly prototyped and thrown away on a regular basis.
+
+As such, concepts may or may not make it to production. In keeping with
+the analogy of a car manufacturer, this may be a design drawing for a
+new car (up to and including showing off a prototype on a trade show),
+or some new technology (stage 2 artificial intelligence).
+
+For Kolab Groupware, this stage mostly involves research and
+development not necessarily based on consumer input.
+
+The output of this process (if at all "valid") is additional input to
+the next stage, :ref:`developer-guide-process-stage-pre-production`.
+
+.. _developer-guide-process-stage-pre-production:
+
+Pre-Production
+--------------
+
+The pre-production stage is an interaction between consumers of the
+product, `Product Owners`_, `Architecture & Design`_ and
+`Quality Assurance`_.
+
+:ref:`developer-guide-process-roles-consumers` of the product request
+features or report issues, that end up on the *Product Backlog* [#]_.
+`Product Owners`_ prioritize these tasks.
+
+Depending on the complexity of the task, and with the existing need to
+facilitate the smoothest possible execution of the **Production** phase,
+`Product Owners`_ and `Scrum Masters`_ request a review to be aided by
+the `Architecture & Design`_ and `Quality Assurance`_ teams.
+
+The input for this stage may be anything from UI elements with the wrong
+teint (easy to forward to **Production** as-is) to complex enhancements
+to Kolab Groupware as a whole, requiring design and possibly touching
+the codebase of multiple software projects (such as the implementation
+of Federated Central Authentication Services with Two-Factor
+Authentication support).
+
+Regardless, the effort at this stage involves breaking down tasks in to
+digestible sub-tasks ready for **Production**.
+
+Production
+----------
+
+Production is the stage where development work takes place. The form in
+which is this happens is through `Sprints`_, which are limited chunks of
+time in which several events take place that shape the development
+process for :ref:`developer-guide-process-roles-developers`.
+
+The input for this stage has initially been composed during the previous
+sprint, with `Product Owners`_ putting tasks from dozens of software
+development project backlogs on to the backlog for the current sprint.
+
+Note that at this point, most facets of tasks have already been
+negotiated -- but only to a certain extent.
+
+If a relatively small task involves changes in one software project
+dependent on changes in another software project, then this should have
+been settled either before the individual sub-tasks end up on the sprint
+backlog, or happens during the sprint planning meeting.
+
+Post-Production
+---------------
+
+Roles
+=====
+
+.. _developer-guide-process-roles-stakeholders:
+
+Stakeholders
+------------
+
+Stakeholders are parties interested in the product, and are responsible
+for the creation of feature implementation requests, bug reports and
+acceptance testing.
+
+As such, stakeholders include customers (of Kolab Systems and its
+partners), but actually include all consumers of Kolab. In other words,
+regardless of affiliation or status, while you are reading this you
+are probably a stakeholder.
+
+However, some stakeholders' wishes have more weight than others. This
+is likely reflected (at most) in the priority of the task(s) at hand,
+which dictates the order in which tasks are worked on.
+
+.. _developer-guide-process-role-product-owner:
+
+Product Owner
+-------------
+
+The role of a product owner (usually a singular entity) is to determine
+the value of goals (set out as epics and tasks in `Maniphest`_) and
+prioritize the associated epics and tasks accordingly.
+
+When the product owner does so, he or she determines the value of goals
+on behalf of the stakeholders, and in light of an overall vision on
+where the product should go.
+
+Within the Kolab Groupware universe, where the *product* is the Kolab
+release, and dozens of individual software development projects make up
+the *product*, several aspects make the `Product Owners`_ a team of
+individuals rather than a single person:
+
+* Not all software development projects are related to the product.
+
+ Such may include work on Phabricator or the Sprint extension to
+ Phabricator (i.e. facilitatory third-party software).
+
+* Not all software development projects are under the control of the
+ business.
+
+ This may include third-party software development projects we
+ contribute to (i.e. Roundcube).
+
+However in the context of `Scrum`_ and particularly the `Sprints`_,
+there is only one stakeholder representative -- **the** Product Owner.
+
+.. _developer-guide-process-roles-scrum-masters:
+
+Scrum Masters
+-------------
+
+`Scrum Masters`_ facilitate the Scrum process ...
+
+.. rubric:: Checklist for the Beginning and End of a Sprint
+
+#. Remove the hash-tag `sprint_server_current` from the past sprint,
+#. Remove the hash-tag `sprint_server_next` from the current sprint,
+#. Add the hash-tag `sprint_server_current` to the current sprint,
+#. Create a new sprint,
+#. Set the start and end dates for the new sprint,
+#. Add the hash-tag `sprint_server_next` to the next sprint.
+
+.. _developer-guide-process-roles-architecture-and-design:
+
+Architecture & Design
+---------------------
+
+.. _developer-guide-process-roles-quality-assurance:
+
+Quality Assurance
+-----------------
+
+.. _developer-guide-process-roles-developers:
+
+Developers
+----------
+
+.. _developer-guide-process-roles-consumers:
+
+Consumers
+---------
+
+* A user or customer requests the resolution of a certain issue.
+
+* The product owner(s)
+
+Continuous Integration, Delivery & Deployment
+=============================================
+
+.. graphviz::
+
+ digraph {
+ rankdir = LR;
+ splines = true;
+ overlab = prism;
+
+ edge [color=gray50, fontname=Calibri, fontsize=11];
+ node [shape=record, fontname=Calibri, fontsize=11];
+
+ "contributor" -> "Phabricator";
+
+ "Phabricator" -> "Build Host";
+
+ "Build Host" -> "Docker Container(s)";
+
+ }
+
+#. A *contributor* contributes code via `Differential`_ to create or
+ update a code review proposal, or `Diffusion`_ to commit directly
+ to the GIT repository for the Software Development Project.
+
+#. `Herald`_, the `Phabricator`_ rule engine, is used to trigger the
+ execution of a `Harbormaster`_ build plan [#]_.
+
+#. `Harbormaster`_ allocates a `Drydock`_ host to execute a script on.
+
+#. The `Drydock`_ host's script is supplied with the repository and
+ the commit in question.
+
+#. The `Drydock`_ hosts used are `Atomic`_ hosts, meaning they run
+ `Docker`_ containers to execute Continuous Integration.
+
+#. The so-called entrypoint for the Docker images (executed when a
+ container is started from the image) is designed to execute
+ Continuous Integration and Continuous Delivery.
+
+Continuous Integration
+----------------------
+
+Continuous Integration is performed using the following stages;
+
+* A *build* stage in which the software build dependencies are
+ installed in to the Docker container, and the software is built.
+
+ In the case of software that does not need to be compiled, such as
+ PHP and/or JavaScript, the *build* stage executes syntax checks.
+
+* A *unit testing* stage, which is limited to testing individual
+ functions of the software, such as ``sum(a, b)`` with ``a=1; b=2``
+ indeed resulting in ``3``, but also does not yield unexpected
+ behaviour:
+
+ * ``sum(a, b)`` over ``a=1`` correctly raises an argument
+ exception (if expected to), or returns ``1`` (if expected to),
+
+ * ``sum(a, b)`` over ``a="Hello"`` and ``b="world!"``;
+
+ * does not unintentionally result in ``"Hello world!"`` nor
+ ``"Helloworld!"``,
+
+ * raises the correct exception, or returns false, none or the
+ equivalent of ``NaN``,
+
+ * ``sum(a, b)`` over ``a=1; b="apple"`` does not result in ``1``
+ (unless expected) and raises the correct exception (if expected
+ to).
+
+ * *etcetera*, *etcetera*.
+
+ Unit testing further includes testing if underlying software
+ behaves in a way that the software can handle correctly. This may
+ include tests that determine whether stripping invalid characters
+ from a string (by **iconv**) yields the expected result, and
+ whether the version of the API provided by underlying software is
+ sufficient.
+
+ .. NOTE::
+
+ In cases where function calls require infrastructure (such as a
+ SQL server) with :term:`fixtures`, unit tests have to replace
+ the code that would interact with the infrastructure with code
+ that returns data as if the infrastructure existed. This is
+ called *mocking up*.
+
+ Tests that do indeed require infrastructure, and/or cannot be
+ *mocked up*, are called *functional tests*.
+
+* A *functional testing* stage, in which functions of the software
+ are tested which require a minimal amount of infrastructure.
+
+ This may include functions that imply a caching layer such as
+ **memcached** is available (populating a cache with fixtures,
+ marking it as dirty, and expecting retrieval to return no data).
+
+ .. NOTE::
+
+ It should be noted that the distinction between *unit* and
+ *functional* testing is largely a matter of job control.
+
+ It should also be noted that functional tests are executed
+ against a *minimal* infrastructure -- the keyword being
+ minimal. This usually precludes infrastructure needed to
+ authenticate users, and usually precludes testing functions
+ against various implementations of a piece of functionality
+ (such as an IMAP server needed by Roundcube testing possibly
+ being either of UW-IMAP, Courier, Cyrus IMAP, Dovecot, ...).
+
+* A *packaging* stage, in which the software is packaged up for
+ distribution.
+
+* An *installation testing* stage, in which the result of the
+ *packaging* is installed.
+
+* An *upgrade testing* stage, in which any number of former release's
+ fixtures are put in place, and the upgrade path is supposed to be
+ completed successfully automatically (such as a
+ :command:`rake db:migrate`).
+
+* An *integration testing* stage
+
+* An *acceptance testing* stage, in which human beings experience
+ the implementation.
+
+Continuous Delivery
+-------------------
+
+.. _developer-guide-using-arcanist:
+
+Using Arcanist
+==============
+
+.. rubric:: Footnotes
+
+.. [#]
+
+ Kolab Groupware consists of dozens of individual product backlogs.
+
+.. [#]
+
+ The sources for this script live in the Stick repository at
+ `drydocker.sh`_.
+
+.. _Scrum: http://en.wikipedia.org/wiki/Scrum_%28software_development%29
+.. _drydocker.sh: https://git.kolab.org/diffusion/QA/browse/master/drydocker.sh

File Metadata

Mime Type
text/plain
Expires
Fri, Apr 3, 8:03 PM (2 d, 22 h ago)
Storage Engine
blob
Storage Format
Raw Data
Storage Handle
18826230
Default Alt Text
D89.1775246591.diff (17 KB)

Event Timeline