1. install necessary software, mentioned in Prerequisites
2. checkout of story_line2_build repo with all submodules somewhere on machine
3. edit options (deploy_dir, data_dir, and logs_dir) in docker-development.properties in repo root directory
4. run ./gradlew prepareStand
5. go to the directory from deploy_dir options
6. run ./docker_up_clean.sh from that dir
7. wait for some period of time
8. ….
9. All ports from docker-development.properties are working and ready for work

---

Table of content:

• Home
• Prerequisites
• Developement stand provisioning
• Components
  • Crawler (Spring Boot, Java)
  • Message Broker (Kafka)
  • Distributed file storage (Hadoop’s HDFS)
  • Data warehouse (Apache Hive)
  • Distributed business logic cluster (Akka, Scala)
  • Distributed data processing cluster (Spark, Scala)
  • Indexing/Search engine (Elasticsearch)
  • REST server (Lagom, Scala)
  • Service coordination (Zookeeper)
  • Time-series database (monitoring data) (InfluxDB)
  • Metrics collector (Telegraf)
  • Monitoring visualization service (Grafana)
  • Reverse proxy/load-balancer (nginx)
• Monitoring Links
• Development
  • Build Tool
  • Provisioning (DevOps specific)
  • Configuration
  • Continuous Integration, Continuous Deployment
  • Implementation design
  • Logging
  • Metrics
  • Used ports
  • Adding new server