Page MenuHomePhorge
Files F117758575

No OneTemporary
Actions

Authored By
Unknown
Size
21 KB
Referenced Files
None
Subscribers
None

View Options

diff --git a/docs/README.md b/docs/README.md
index d0e89aa..5f06f6c 100644
--- a/docs/README.md
+++ b/docs/README.md
@@ -1,90 +1,90 @@
This is the helm managed kubernetes deployment of kolab.
By default this will deploy a local k3s instance, but the templates should be compatible with any kubernetes distribution.
The default deployment will be completely self-contained, special care will need to be taken if external services are integrated (such as an existing mysql instance).
# Getting started
## Requirements
* bash (from your distro)
* openssl (from your distro)
* uuidgen (from your distro)
* kubectl: https://kubernetes.io/docs/tasks/tools/install-kubectl-linux/
* helm: https://helm.sh/docs/intro/install/
-* Secrets: pull secret, username, password
+* Secrets: quay.io/apheleiait username + password
### K3s requirements
* k3s requirements: https://docs.k3s.io/installation/requirements
* disabled firewall (`systemctl stop firewalld`, I suppose it's possible to configure the firewall properly, but by default your cluster will fail in all sorts of interesting ways that are hard to troubleshoot.)
* stop conflicting docker deployments that are using the same ports
## Quickstart
-To just deploy a k3s deployment locally, run this.
+To just deploy a k3s deployment locally, run this (you will requires a username and password for registry access).
./kolabctl configure
- ./kolabctl deploy --k3s
+ ./kolabctl deploy --k3s $username $password
## kolabctl
The local deployment can be managed using kolabctl which simplifies various administrative tasks.
kolabctl is a thin wrapper around helm/kubectl/artisan, so it is not required to use kolabctl if you are more familiar with the underlying tools,
but it illustrates the various tasks.
## Deployment
### Configure
The deployment configuration is entirely contained in the values.yaml file.
-This step will require your pull secret
+This step will require your quay.io username and password
To generate an initial values.yaml file:
./kolabctl configure
you can pass in all requested values like so:
- env PUBLIC_IP=1.2.3.4 DOMAIN=kolab.local ADMIN_PASSWORD=simple123 PULL_SECRET="s1ob+PYs0b1nc9wMcy213tJFTGhmLWazkEwRmQkuKhsw0YriNSfFzEPpZN4cyCCUkEqXiK54n32kl3t8w6WIR0B9I8ENg0CnhmqCgfkHb8v2B7r4o8xKUOULOSWLQJe3jMa4J39UIkAFMfy/YoENJuVODLeC5Df9t2RtZJcYj3M=" ./kolabctl configure
+ env PUBLIC_IP=1.2.3.4 DOMAIN=kolab.local ADMIN_PASSWORD=simple123 USERNAME="quay.io username" PASSWORD="quay.io password" ./kolabctl configure
Adjust the values as desired before deploying, but please not that you can easily break the deployment
by deviating from the defaults.
The following values are required during setup:
* PUBLIC_IP: Public ip of the deployment
* ADMIN_PASSWORD: The password for the admin@$DOMAIN user
-* PULL_SECRET: The docker secret for the quay.io/apheleiait registry.
+* USERNAME/PASSWORD: Username and password for the quay.io/apheleiait registry.
Other values can be adjusted in the values.yaml file before deployment, and generated secrets can be inspected inside the values.yaml file.
### Deploy on Kubernetes
./kolabctl deploy
This will deploy Kolab via helm and kubectl on the currently connected cluster.
Once the command completes, run
./kolabctl selfcheck
to validate that the deployment was successful.
#### Deploy including k3s
This step will require your username and password.
./kolabctl deploy --k3s $username $password
This will first install a k3s cluster locally, configure it and install helm.
Next helm will login to the registry via username and password, and the kolab images will be pulled from the registry.
Finally, kolab is deployed in the new k3s instance.
## Updating
To apply the latest changes from your local values.yaml with the latest chart changes, run:
./kolabctl update
diff --git a/kolabctl b/kolabctl
index c98365b..375b116 100755
--- a/kolabctl
+++ b/kolabctl
@@ -1,572 +1,591 @@
#!/bin/bash
# shellcheck disable=SC2068
# shellcheck disable=SC2086
HOST=${HOST:-kolab.local}
NAMESPACE=${NAMESPACE:-kolab}
CHART_VERSION=${CHART_VERSION:-"0.1.0"}
CHART_URL=${CHART_URL:-"oci://quay.io/apheleiait/kolab/kolab"}
export KUBECONFIG=${KUBECONFIG:-"$HOME/.kube/config"}
### ========== Implementation Details ============
#Ensure we are connected to the correct cluster (localhost only for now)
__ensure_server() {
if ! grep server: ~/.kube/config | grep 127.0.0.1; then
echo "Connected to the wrong server!"
exit
fi
}
__helm_upgrade() {
if [[ -f values.yaml ]]; then
ARGS="-f values.yaml"
fi
time helm upgrade -i kolab --create-namespace $ARGS $@ --namespace "$NAMESPACE" "$CHART_URL" --version "$CHART_VERSION"
command kubectl -n $NAMESPACE rollout status deployment
command helm status kolab -n $NAMESPACE
}
__helm_template() {
if [[ -f values.yaml ]]; then
ARGS="-f values.yaml"
fi
command helm template kolab --namespace "$NAMESPACE" "$CHART_URL" $ARGS $@ --version "$CHART_VERSION"
}
__k3s_login() {
# Copy k3s config
mkdir -p "$HOME/.kube"
sudo cp /etc/rancher/k3s/k3s.yaml "$HOME/.kube/config"
sudo chown $(id -u):$(id -g) "$HOME/.kube/config"
chmod 600 "$HOME/.kube/config"
echo "Export this: export KUBECONFIG=$HOME/.kube/config"
}
# Pull images as defined in helm chart.
# Pass in username and password for registry
# This currently only pulls the kolab images (for the rest we'd have to figure out where to pull from)
__k3s_pull() {
# Figure out which images to pull
images=$(__helm_template | grep "image: " | grep -v "#" | grep -oP "(?<=image: ).*" | sed 's/\"//g' | uniq )
for image in $images; do
echo "Pulling $image"
# For the quay.io/apheleiait images we need to pass in the credentials
if [[ $image == *"quay.io/apheleiait"* ]]; then
if [[ -z $1 || -z $2 ]]; then
echo "You need to specify the quay.io/apheleiait username and password"
exit 1
fi
sudo k3s ctr images pull --user "$1:$2" "$image"
else
#TODO Support non apheleiait images?
sudo k3s ctr images pull "$image"
fi
done
}
__k3s_install() {
# TURN OFF firewalld if you're seeing any networking issues.
# Install K3S
sudo mkdir -p /etc/rancher/k3s/
curl -sfL https://get.k3s.io | INSTALL_K3S_EXEC="server" sh -s - --disable traefik,servicelb
sudo chmod 644 /etc/rancher/k3s/k3s.yaml
curl https://raw.githubusercontent.com/helm/helm/main/scripts/get-helm-3 | bash
__k3s_login
# Check K3S
kubectl get pods -n kube-system
# Install certmanager
kubectl apply -f https://github.com/cert-manager/cert-manager/releases/download/v1.13.3/cert-manager.crds.yaml
helm repo add jetstack https://charts.jetstack.io
helm repo add haproxytech https://haproxytech.github.io/helm-charts
helm repo add prometheus-community https://prometheus-community.github.io/helm-charts
helm repo add metallb https://metallb.github.io/metallb
helm repo update
helm install cert-manager jetstack/cert-manager \
--wait \
--namespace cert-manager \
--create-namespace \
--version v1.13.3
helm install haproxy-ingress haproxytech/kubernetes-ingress \
--wait \
--namespace haproxy-controller \
--create-namespace \
--set controller.config.ssl-certificate=kolab/kolab-cert \
--set controller.kind=DaemonSet \
--set controller.daemonset.useHostPort=true \
--version 1.38.5
helm install kube-state-metrics prometheus-community/kube-state-metrics \
--wait \
--namespace default \
--version 3.0.0
helm install metallb metallb/metallb --create-namespace \
--namespace metallb-system --wait
}
__sum_memory_requests() {
res=$(kubectl get pods -o=jsonpath='{.items[*]..resources.requests.memory}' -n "$NAMESPACE")
let tot=0
for i in $res; do
if [[ $i =~ "Mi" ]]; then
i=$(echo $i | sed 's/[^0-9]*//g')
tot=$(( tot + i ))
elif [[ $i =~ "Gi" ]]; then
i=$(echo $i | sed 's/[^0-9]*//g')
tot=$(( tot + i * 1000 ))
fi
done
echo $tot
}
__sum_cpu_requests() {
res=$(kubectl get pods -o=jsonpath='{.items[*]..resources.requests.cpu}' -n "$NAMESPACE")
let tot=0
for i in $res; do
if [[ $i =~ "m" ]]; then
i=$(echo $i | sed 's/[^0-9]*//g')
tot=$(( tot + i ))
else
tot=$(( tot + i*1000 ))
fi
done
echo $tot
}
__replace_multiline() {
# Echo content, indent by the passed in spaces, insert before matching line
echo -n "$2" | sed -e "s/^/$4/" | sed -i "/$1/e cat /dev/stdin;echo" $3
# delete the matching line
sed -i "/$1/d" $3
}
### ========== End of Implementation ============
#
### ========== Commands ============
# Cluster admin commands
kolab__deploy() {
if [[ ! -f values.yaml ]]; then
echo "Missing a values.yaml file, run configure first."
exit 1
fi
if [[ "$1" == "--k3s" ]]; then
shift
__k3s_install
__k3s_pull $1 $2
if [[ -n $1 && -n $2 ]]; then
helm registry login quay.io/apheleiait -u "$1" -p "$2"
fi
fi
__ensure_server
echo "This is going to take a while (timeout after 30 minutes)"
__helm_upgrade
}
kolab__uninstall() {
command helm uninstall kolab --wait --timeout 10m0s --debug $ARGS $@ --namespace "$NAMESPACE"
}
kolab__update() {
__helm_upgrade $@
}
kolab__template() {
__helm_template $@
}
kolab__pull() {
__k3s_pull $@
}
# Update the kolabctl script from the latest tarball
kolab__selfupdate() {
wget https://mirror.apheleia-it.ch/pub/kolab-kubernetes-latest.tar.gz -O /tmp/kolab-kubernetes-latest.tar.gz
tar -zxvf /tmp/kolab-kubernetes-latest.tar.gz kolabctl
}
# kolab__backup() {
# command bin/backup.sh
# }
# kolab__restore() {
# command bin/restore.sh
# }
# Render a values.yaml file from a template in docs
kolab__configure() {
TEMPLATE="docs/k3s.values.yaml"
if [[ -f values.yaml ]]; then
echo "values.yaml already exists";
exit 1
fi
if [[ -z $DOMAIN ]]; then
echo "Please enter your domain:"
read -r DOMAIN
fi
if [[ -z $ADMIN_PASSWORD ]]; then
echo "Please enter your new admin password for the admin@$DOMAIN user:"
read -r ADMIN_PASSWORD
fi
- if [[ -z $PULL_SECRET ]]; then
- echo "Please enter your registry access secret:"
- read -r PULL_SECRET
+ if [[ -z $USERNAME ]]; then
+ echo "Please enter your registry access username:"
+ read -r USERNAME
fi
+ if [[ -z $PASSWORD ]]; then
+ echo "Please enter your registry access password:"
+ read -r USERNAME
+ fi
+
+ # Assamble the docker secret, which will be our pull secret
+ PULL_SECRET=$(
+cat <<EOF | base64 -w0
+{
+ "auths": {
+ "quay.io": {
+ "auth": "$(echo -n "$USERNAME:$PASSWORD" | base64 -w0)",
+ "email": ""
+ }
+ }
+}
+EOF
+)
+
if [[ -z $PUBLIC_IP ]]; then
echo "Please enter your public ip:"
read -r PUBLIC_IP
fi
TEST_PASSWORD=$(openssl rand -base64 24)
APP_KEY=$(openssl rand -base64 32);
APP_KEY="base64:${APP_KEY}"
COTURN_STATIC_SECRET=$(openssl rand -hex 32);
PASSPORT_PROXY_OAUTH_CLIENT_ID=$(uuidgen);
PASSPORT_PROXY_OAUTH_CLIENT_SECRET=$(openssl rand -base64 32)
PASSPORT_PRIVATE_KEY=$(openssl genrsa 4096);
PASSPORT_PUBLIC_KEY=$(echo "$PASSPORT_PRIVATE_KEY" | openssl rsa -pubout 2>/dev/null)
MEET_WEBHOOK_TOKEN=$(openssl rand -hex 32)
MEET_SERVER_TOKEN=$(openssl rand -hex 32)
DB_ROOT_PASSWORD=simple123
DB_KOLAB_PASSWORD=simple123
DB_LEGACY_PASSWORD=simple123
DB_ROUNDCUBE_PASSWORD=simple123
REDIS_PASSWORD=simple123
MAIL_NOREPLY_PASSWORD=simple123
MINIO_ROOT_PASSWORD=simple123
IMAP_ADMIN_PASSWORD=simple123
ROUNDCUBE_DES_KEY=$(openssl rand -base64 24);
DKIM_IDENTIFIER="dkim20240318"
DKIM_KEY=$(openssl genrsa 2048);
cp "$TEMPLATE" values.yaml
# / can appear in base64, but | not
sed -i \
-e "s|DOMAIN|${DOMAIN}|g" \
-e "s|TEST_PASSWORD|${TEST_PASSWORD}|g" \
-e "s|IMAP_ADMIN_PASSWORD|${IMAP_ADMIN_PASSWORD}|g" \
-e "s|APP_KEY|${APP_KEY}|g" \
-e "s|PULL_SECRET|${PULL_SECRET}|g" \
-e "s|PASSPORT_PROXY_OAUTH_CLIENT_ID|${PASSPORT_PROXY_OAUTH_CLIENT_ID}|g" \
-e "s|PASSPORT_PROXY_OAUTH_CLIENT_SECRET|${PASSPORT_PROXY_OAUTH_CLIENT_SECRET}|g" \
-e "s|PASSPORT_PROXY_OAUTH_CLIENT_SECRET|${PASSPORT_PROXY_OAUTH_CLIENT_SECRET}|g" \
-e "s|ROUNDCUBE_DES_KEY|${ROUNDCUBE_DES_KEY}|g" \
-e "s|MEET_WEBHOOK_TOKEN|${MEET_WEBHOOK_TOKEN}|g" \
-e "s|MEET_SERVER_TOKEN|${MEET_SERVER_TOKEN}|g" \
-e "s|COTURN_STATIC_SECRET|${COTURN_STATIC_SECRET}|g" \
-e "s|DB_ROOT_PASSWORD|${DB_ROOT_PASSWORD}|g" \
-e "s|DB_KOLAB_PASSWORD|${DB_KOLAB_PASSWORD}|g" \
-e "s|DB_LEGACY_PASSWORD|${DB_LEGACY_PASSWORD}|g" \
-e "s|DB_ROUNDCUBE_PASSWORD|${DB_ROUNDCUBE_PASSWORD}|g" \
-e "s|REDIS_PASSWORD|${REDIS_PASSWORD}|g" \
-e "s|MAIL_NOREPLY_PASSWORD|${MAIL_NOREPLY_PASSWORD}|g" \
-e "s|MINIO_ROOT_PASSWORD|${MINIO_ROOT_PASSWORD}|g" \
-e "s|DKIM_IDENTIFIER|${DKIM_IDENTIFIER}|g" \
-e "s|ADMIN_PASSWORD|${ADMIN_PASSWORD}|g" \
-e "s|PUBLIC_IP|${PUBLIC_IP}|g" \
values.yaml
__replace_multiline PASSPORT_PRIVATE_KEY "$PASSPORT_PRIVATE_KEY" values.yaml " "
__replace_multiline PASSPORT_PUBLIC_KEY "$PASSPORT_PUBLIC_KEY" values.yaml " "
__replace_multiline DKIM_KEY "$DKIM_KEY" values.yaml " "
}
kolab__registry_login() {
helm registry login quay.io/apheleiait -u "$1" -p "$2"
}
__wait_for_service() {
echo "Waiting for service/$1"
# Wait for the ingress section with the assigned ip to appear
command timeout 10s bash -c "until kubectl -n kolab get service/$1 --output=jsonpath='{.status.loadBalancer}' | grep 'ingress'; do : ; done" || :
if command kubectl -n kolab get service/external-proxy --output=jsonpath='{.status.loadBalancer}' | grep "ingress"; then
echo "Service $1 is bound";
else
echo "Service $1 does not have an external ip";
exit 1;
fi
}
kolab__selfcheck() {
set -e
if command kubectl -n $NAMESPACE describe secret/kolab-cert-static --request-timeout "5s" &> /dev/null; then
echo "Found static tls secret"
elif command kubectl -n $NAMESPACE describe secret/kolab-cert-letsencrypt --request-timeout "5s" &> /dev/null; then
echo "Found letsencrypt secret"
else
echo "TLS secret not found"
exit 1
fi
# Wait for all rollouts to complete
command kubectl -n $NAMESPACE rollout status deployment
__wait_for_service "external-proxy"
__wait_for_service "external-smtp"
# Check that all pvcs are available
for pvc in $(kubectl -n kolab get --no-headers=true pvc -o name); do
command kubectl -n kolab wait --for=jsonpath='{.status.phase}'=Bound $pvc
done
command helm status kolab -n $NAMESPACE | grep "STATUS: deployed"
command kubectl exec --stdin --tty -n "$NAMESPACE" deployment/imap -- bash -c "testsaslauthd -u \$IMAP_ADMIN_LOGIN -p \$IMAP_ADMIN_PASSWORD"
command kubectl exec --stdin --tty -n "$NAMESPACE" deployment/horizon -- ./artisan status:health --check=Redis --check=IMAP
command kubectl exec --stdin --tty -n "$NAMESPACE" deployment/roundcube -- ./checkconnections.sh
echo "All tests passed"
}
kolab__stop() {
command /usr/local/bin/k3s-killall.sh
}
kolab__start() {
command systemctl start k3s
}
kolab__status() {
echo "# Cluster info"
command kubectl cluster-info
echo "# Helm deployments"
command helm list -A
echo "# Helm status"
command helm status kolab -n "$NAMESPACE" --show-resources
echo "# Events"
command kubectl get events --sort-by=.metadata.creationTimestamp
}
kolab__shell() {
if [[ $1 == "" ]]; then
# command kubectl run kolab -ti --rm -n "$NAMESPACE" --image localhost:5000/webapp:latest -- /bin/bash
command kubectl run utils -ti --rm -n "$NAMESPACE" --image quay.io/apheleiait/kolab/utils:latest -- /bin/bash
else
command kubectl exec --stdin --tty -n "$NAMESPACE" "deployment/$1" -- /bin/bash
fi
}
kolab__logs() {
deployment="$1"
shift
command kubectl -n "$NAMESPACE" logs "deployment/$deployment" $@
}
# Tail the logs via logcli
# --pod=: wildcard match on the pod_name label
# --search=: Run a fulltext query match on the record
# -f: Stream the log
# --full: Full output including the entire record
kolab__tail() {
if [[ "$*" =~ "--pod" ]]; then
ARGUMENT=$(echo "$*" | grep -o -e "--pod=.*" | cut -d ' ' -f 1)
POD_NAME=$(echo "$ARGUMENT" | cut -d '=' -f 2)
POD_FILTER=",pod_name=~\"$POD_NAME.*\""
set -- "$(echo "$*" | sed "s|$ARGUMENT *||")"
fi
if [[ "$*" =~ "--search" ]]; then
ARGUMENT=$(echo "$*" | grep -o -e "--search=.*" | cut -d ' ' -f 1)
SEARCH_STRING=$(echo "$ARGUMENT" | cut -d '=' -f 2)
FULLTEXT_FILTER=" |~ \"$SEARCH_STRING\""
set -- "$(echo "$*" | sed "s|$ARGUMENT *||")"
fi
if [[ "$*" =~ "--full" ]]; then
set -- "$(echo "$*" | sed "s|--full *||")"
else
OUTPUT="--output=raw"
fi
echo "$*"
if [[ "$*" =~ "-f" ]]; then
FOLLOW="--follow"
fi
# The format is --from=2024-02-22T08:30:24Z --to=2024-02-23T12:30:24Z
if [[ "$*" =~ "--from" ]]; then
# Can't have --from without --to
FROM=$(echo "$*" | grep -o -e "--from=.*" | cut -d ' ' -f 1)
TO=$(echo "$*" | grep -o -e "--to=.*" | cut -d ' ' -f 1)
#beware of --limit (it will continue to apply, so you may get partial results)
# Can't follow with a time range
FOLLOW=""
fi
QUERY="{namespace_name=\"$NAMESPACE\"$POD_FILTER} $FULLTEXT_FILTER | json log | line_format \"{{ __timestamp__ }} \t {{.pod_name | trunc 10}} \t{{.log}}\""
echo "Running query: $QUERY"
env LOKI_ADDR="http://$HOST/" logcli query "$QUERY" $OUTPUT $FOLLOW $FROM $TO
}
kolab__describe() {
deployment="$1"
shift
command kubectl -n "$NAMESPACE" describe "deployment/$deployment" $@
}
kolab__restart() {
command kubectl -n "$NAMESPACE" rollout restart deployment
}
kolab__ps() {
command kubectl -n "$NAMESPACE" get pods
}
kolab__get() {
command kubectl -n "$NAMESPACE" get $@
}
kolab__volumes() {
command kubectl -n "$NAMESPACE" get persistentvolumeclaims $@
}
kolab__build() {
command env REGISTRY="localhost:5000" build/build.sh --nochecks
}
kolab__top() {
command kubectl top pods -n "$NAMESPACE" --sum=true
command kubectl get pods -n "$NAMESPACE" -o=jsonpath='{range .items[*]}{"\n"}{.metadata.name}{":\t\t"}{..resources.requests.memory}{end}'
echo
echo
echo "Total memory requests: $(__sum_memory_requests) Mi"
echo "Total cpu requests: $(__sum_cpu_requests) m"
}
kolab__metrics() {
command kubectl get --raw /apis/metrics.k8s.io/v1beta1/pods | jq
command kubectl get --raw /apis/metrics.k8s.io/v1beta1/nodes | jq
# # Get the metrics for node <node_name>
# kubectl get --raw /apis/metrics.k8s.io/v1beta1/<node_name> | jq '.'
# # Get the metrics for pode <pod_name>
# kubectl get --raw /apis/metrics.k8s.io/v1beta1/<pod_name> | jq '.'
}
kolab__reset() {
if [[ "$1" == "--force" ]]; then
REPLY="y"
else
read -p "Are you sure? This will delete the k3s cluster including all data" -n 1 -r
echo
fi
if [[ "$REPLY" =~ ^[Yy]$ ]];
then
/usr/local/bin/k3s-uninstall.sh
fi
}
kolab__login() {
if [[ "$1" == "--k3s" ]]; then
__k3s_login
else
echo "Configure kubectl manually, login is not implemented"
fi
command kubectl cluster-info
}
kolab__exec() {
container=$1
shift
command kubectl exec --stdin --tty -n "$NAMESPACE" "deployment/$container" -- $@
}
# Administration commands
kolab__cyradm() {
# FIXME grep credentials from files
command kubectl exec --stdin --tty -n "$NAMESPACE" deployment/imap -- cyradm --auth PLAIN -u admin@kolab.local -w simple123 --port 11143 localhost
}
kolab__reseed() {
read -p "Are you sure? This will delete all data" -n 1 -r
echo
if [[ "$REPLY" =~ ^[Yy]$ ]];
then
kolab__exec horizon ./artisan migrate:fresh --seed
fi
}
kolab__users() {
kolab__exec horizon ./artisan users --attr=email
}
kolab__password() {
kolab__exec horizon ./artisan user:password $@
}
kolab__db() {
kolab__exec mariadb mysql -p
}
kolab__help() {
cat <<EOF
This is the kolab commandline utility.
The following commands are available:
deploy: Deploy kolab
start: Start all containers
stop: Stop all containers
update: This will update all containers.
backup: Create a backup in backup/
restore: Restore a backup from backup/
selfcheck: Run a selfcheck to ensure kolab is functional
EOF
}
cmdname=$1
shift
if [[ "${cmdname}" != "deploy" && "${cmdname}" != "configure" ]]; then
__ensure_server
fi
# make sure we actually *did* get passed a valid function name
if declare -f "kolab__$cmdname" >/dev/null 2>&1; then
"kolab__$cmdname" "${@:1}"
else
echo "Function $cmdname not recognized" >&2
kolab__help
exit 1
fi

File Metadata

Mime Type
text/x-diff
Expires
Sat, Apr 4, 9:51 AM (3 w, 5 d ago)
Storage Engine
blob
Storage Format
Raw Data
Storage Handle
18823548
Default Alt Text
(21 KB)

Event Timeline