- Download spark 2.0.11 how to#
- Download spark 2.0.11 full#
- Download spark 2.0.11 code#
- Download spark 2.0.11 trial#
More projects upcoming, please stay tuned. Schedulis - Workflow Task Scheduling Tool Below are the currently available open-source projects.ĭataSphere Studio - Data Application Integration& Development Framework You can find more detailed architecture docs in Architecture.īased on Linkis the computation middleware, we've built a lot of applications and tools on top of it in the big data platform suite WeDataSphere.
Download spark 2.0.11 how to#
You can find examples and guidance for how to use and manage Linkis in User_Manual, Engine_Usage_Documents and API_Documents. Please refer to Deployment_Documents to do the deployment. Please follow Compile Guide to compile Linkis from source code.
Download spark 2.0.11 code#
Please go to the Linkis releases page to download a compiled distribution or a source code package of Linkis. Supports FlinkSQL code, and also supports Flink Jar to Linkis Manager to start a new Yarn application.ĮlasticSearch EngineConn. System and user-level unified material management, which can be shared and transferred across users and systems.įlink EngineConn. Set in one place, automatic reference everywhere
Generate Context ID for each task/request, associate and manage user and system resource files (JAR, ZIP, Properties, etc.), result set, parameter variable, function, etc., across user, system, and computing engine. ResourceManager is not only capable of managing resources for Yarn and Linkis EngineManger as in Linkis 0.X, but also able to provide label-based multi-level resource allocation and recycling, allowing itself to have powerful resource management capabilities across mutiple Yarn clusters and mutiple computation resource types As a computation middleware, it will receive, execute and manage tasks and requests for various computation storage engines, including batch tasks, interactive query tasks, real-time streaming tasks and storage tasks
Download spark 2.0.11 full#
Support full stack computation/storage engine. With services such as Orchestrator, Label Manager and customized Spring Cloud Gateway, Linkis is able to provide multi-level labels based, cross-cluster/cross-IDC fine-grained routing, load balance, multi-tenancy, traffic control, resource control, and orchestration strategies like dual-active, active-standby, etc. Powerful task/request governance capabilities.
Supported scripting languages: SparkSQL, HiveQL, Python, Shell, Pyspark, R, Scala and JDBC, etc. Support for diverse underlying computation storage engines.Ĭurrently supported computation/storage engines: Spark, Hive, Python, Presto, ElasticSearch, MLSQL, TiSpark, JDBC, Shell, etc Ĭomputation/storage engines to be supported: Flink(Supported in version >=1.0.2), Impala, etc Lots of companies have already used Linkis as a unified entrance for the underlying computation and storage engines of the big data platform.
Download spark 2.0.11 trial#
Since the first release of Linkis in 2019, it has accumulated more than 700 trial companies and 1000+ sandbox trial users, which involving diverse industries, from finance, banking, tele-communication, to manufactory, internet companies and so on. By decoupling the application layer and the engine layer, it simplifies the complex network call relationship, and thus reduces the overall complexity and saves the development and maintenance costs as well.
By using standard interfaces such as REST/WS/JDBC provided by Linkis, the upper applications can easily access the underlying engines such as MySQL/Spark/Hive/Presto/Flink, etc., and achieve the intercommunication of user resources like unified variables, scripts, UDFs, functions and resource files at the same time.Īs a computation middleware, Linkis provides powerful connectivity, reuse, orchestration, expansion, and governance capabilities. Linkis builds a layer of computation middleware between upper applications and underlying engines.
0 kommentar(er)
