1.什么是分布式調(diào)度
分布式調(diào)度==》分布式 + 調(diào)度中心 + 執(zhí)行器，調(diào)度中心根據(jù)對應(yīng)任務(wù)執(zhí)行條件去分布式的執(zhí)行任務(wù)。
2.什么時候需要使用分布式調(diào)度
當在單機的情況下，我們的任務(wù)執(zhí)行服務(wù)，無法滿足任務(wù)處理的時候，我們就需將執(zhí)行服務(wù)進行集群，這個時候我們就需要使用分布式調(diào)度協(xié)調(diào) 各個服務(wù)從而是任務(wù)有條不紊的執(zhí)行。
3.分布式調(diào)度框架有哪些？
常見的分布式調(diào)度框架有：cronsun、Elastic-job、saturn、its、TBSchedule、xxl-job等
這里我整合了xxl-job,xx-job比較輕量級而且簡單易學
4.整合xxl-job
下載xxl-job源碼：https://github.com/xuxueli/xxl-job/
解壓后使用idea打開項目，
a.執(zhí)行sql : \xxl-job-master\doc\db\tables_xxl_job.sql
b.修改配置： \xxl-job-master\xxl-job-admin\src\main\resources\application.properties
####web port
server.port=8080
server.servlet.context-path=/xxl-job-admin
### actuator
management.server.servlet.context-path=/actuator
management.health.mail.enabled=false
### resources
spring.mvc.servlet.load-on-startup=0
spring.mvc.static-path-pattern=/static/**
spring.resources.static-locations=classpath:/static/
### freemarker
spring.freemarker.templateLoaderPath=classpath:/templates/
spring.freemarker.suffix=.ftl
spring.freemarker.charset=UTF-8
spring.freemarker.request-context-attribute=request
spring.freemarker.settings.number_format=0.##########
### mybatis
mybatis.mapper-locations=classpath:/mybatis-mapper/*Mapper.xml
#mybatis.type-aliases-package=com.xxl.job.admin.core.model
### xxl-job, datasource
spring.datasource.url=jdbc:mysql://127.0.0.1:3306/xxl_job?useUnicode=true&characterEncoding=UTF-8&autoReconnect=true&serverTimezone=Asia/Shanghai
spring.datasource.username=root
spring.datasource.password=root
spring.datasource.driver-class-name=com.mysql.cj.jdbc.Driver
### datasource-pool
spring.datasource.type=com.zaxxer.hikari.HikariDataSource
spring.datasource.hikari.minimum-idle=10
spring.datasource.hikari.maximum-pool-size=30
spring.datasource.hikari.auto-commit=true
spring.datasource.hikari.idle-timeout=30000
spring.datasource.hikari.pool-name=HikariCP
spring.datasource.hikari.max-lifetime=900000
spring.datasource.hikari.connection-timeout=10000
spring.datasource.hikari.connection-test-query=SELECT 1
spring.datasource.hikari.validation-timeout=1000
### xxl-job, email
spring.mail.host=smtp.qq.com
spring.mail.port=25
spring.mail.username=xxx@qq.com
spring.mail.from=xxx@qq.com
spring.mail.password=123456
spring.mail.properties.mail.smtp.auth=true
spring.mail.properties.mail.smtp.starttls.enable=true
spring.mail.properties.mail.smtp.starttls.required=true
spring.mail.properties.mail.smtp.socketFactory.class=javax.net.ssl.SSLSocketFactory
### 7、管理界面的登錄用戶名密碼
xxl.job.login.username=admin
xxl.job.login.password=123456
### xxl-job, access token
xxl.job.accessToken=
### xxl-job, i18n (default is zh_CN, and you can choose "zh_CN", "zh_TC" and "en")
xxl.job.i18n=zh_CN
## xxl-job, triggerpool max size
xxl.job.triggerpool.fast.max=200
xxl.job.triggerpool.slow.max=100
### xxl-job, log retention days
xxl.job.logretentiondays=30
c.啟動xxl-job-admin，訪問地址為: http://localhost:8080/xxl-job-admin
image.png
d.啟動執(zhí)行器： \xxl-job-master\xxl-job-executor-samples
或者自己實現(xiàn)執(zhí)行器：
1.新建一個空的maven項目，引入maven依賴：
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-web</artifactId>
</dependency>
<dependency>
<groupId>com.xuxueli</groupId>
<artifactId>xxl-job-core</artifactId>
<version>2.3.0-SNAPSHOT</version>
</dependency>
- 配置application.properties
# web port
server.port=8081
# no web
#spring.main.web-environment=false
# log config
logging.config=classpath:logback.xml
### xxl-job admin address list, such as "http://address" or "http://address01,http://address02"
xxl.job.admin.addresses=http://127.0.0.1:8080/xxl-job-admin
### xxl-job, access token
xxl.job.accessToken=
### xxl-job executor appname
xxl.job.executor.appname=xxl-job-executor-rj
### xxl-job executor registry-address: default use address to registry , otherwise use ip:port if address is null
xxl.job.executor.address=
### xxl-job executor server-info
xxl.job.executor.ip=
xxl.job.executor.port=9999
### xxl-job executor log-path
xxl.job.executor.logpath=/data/applogs/xxl-job/jobhandler
### xxl-job executor log-retention-days,值為-1表示永不刪除
xxl.job.executor.logretentiondays=30
3.執(zhí)行器配置 XxlJobConfig ：
/**
* Create by YSOcean
*/
@Configuration
public class XxlJobConfig {
private Logger logger = LoggerFactory.getLogger(XxlJobConfig.class);
@Value("${xxl.job.admin.addresses}")
private String adminAddresses;
@Value("${xxl.job.executor.appname}")
private String appName;
@Value("${xxl.job.executor.ip}")
private String ip;
@Value("${xxl.job.executor.port}")
private int port;
@Value("${xxl.job.accessToken}")
private String accessToken;
@Value("${xxl.job.executor.logpath}")
private String logPath;
@Value("${xxl.job.executor.logretentiondays}")
private int logRetentionDays;
@Bean(initMethod = "start", destroyMethod = "destroy")
public XxlJobSpringExecutor xxlJobExecutor() {
logger.info(">>>>>>>>>>> xxl-job config init,port：{}",port);
XxlJobSpringExecutor xxlJobSpringExecutor = new XxlJobSpringExecutor();
xxlJobSpringExecutor.setAdminAddresses(adminAddresses);
xxlJobSpringExecutor.setAppname(appName);
xxlJobSpringExecutor.setIp(ip);
xxlJobSpringExecutor.setPort(port);
xxlJobSpringExecutor.setAccessToken(accessToken);
xxlJobSpringExecutor.setLogPath(logPath);
xxlJobSpringExecutor.setLogRetentionDays(logRetentionDays);
return xxlJobSpringExecutor;
}
}
4.執(zhí)行器：
/**
* 任務(wù)Handler示例（Bean模式）
*
* 開發(fā)步驟：
* 1、繼承"IJobHandler"：“com.xxl.job.core.handler.IJobHandler”；
* 2、注冊到Spring容器：添加“@Component”注解，被Spring容器掃描為Bean實例；
* 3、注冊到執(zhí)行器工廠：添加“@JobHandler(value="自定義jobhandler名稱")”注解，注解value值對應(yīng)的是調(diào)度中心新建任務(wù)的JobHandler屬性的值。
* 4、執(zhí)行日志：需要通過 "XxlJobLogger.log" 打印執(zhí)行日志；
*/
@Component
public class JobHandlerDemo{
private Logger logger = LoggerFactory.getLogger(JobHandlerDemo.class);
/**
* 1、簡單任務(wù)示例（Bean模式）
*/
@XxlJob("demoJobHandler")
public void demoJobHandler() throws Exception {
XxlJobHelper.log("XXL-JOB, Hello World.");
//執(zhí)行業(yè)務(wù)邏輯
// default success
}
/**
* 2、分片廣播任務(wù)
*/
@XxlJob("shardingJobHandler")
public void shardingJobHandler(String param) throws Exception {
// 分片參數(shù)
int shardIndex = XxlJobHelper.getShardIndex();
int shardTotal = XxlJobHelper.getShardTotal();
logger.info("shardingJobHandler() ,port：{},param:{}",9997,param);
XxlJobHelper.log("分片參數(shù)：當前分片序號 = {}, 總分片數(shù) = {},port:{}", shardIndex, shardTotal,9997);
// 業(yè)務(wù)邏輯
for (int i = 0; i < shardTotal; i++) {
if (i == shardIndex) {
XxlJobHelper.log("第 {} 片, 命中分片開始處理", i);
} else {
XxlJobHelper.log("第 {} 片, 忽略", i);
}
}
}
/**
* 3、命令行任務(wù)
*/
@XxlJob("commandJobHandler")
public void commandJobHandler() throws Exception {
String command = XxlJobHelper.getJobParam();
int exitValue = -1;
BufferedReader bufferedReader = null;
try {
// command process
ProcessBuilder processBuilder = new ProcessBuilder();
processBuilder.command(command);
processBuilder.redirectErrorStream(true);
Process process = processBuilder.start();
//Process process = Runtime.getRuntime().exec(command);
BufferedInputStream bufferedInputStream = new BufferedInputStream(process.getInputStream());
bufferedReader = new BufferedReader(new InputStreamReader(bufferedInputStream));
// command log
String line;
while ((line = bufferedReader.readLine()) != null) {
XxlJobHelper.log(line);
}
// command exit
process.waitFor();
exitValue = process.exitValue();
} catch (Exception e) {
XxlJobHelper.log(e);
} finally {
if (bufferedReader != null) {
bufferedReader.close();
}
}
if (exitValue == 0) {
// default success
} else {
XxlJobHelper.handleFail("command exit value("+exitValue+") is failed");
}
}
/**
* 4、跨平臺Http任務(wù)
* 參數(shù)示例：
* "url: http://www.baidu.com\n" +
* "method: get\n" +
* "data: content\n";
*/
@XxlJob("httpJobHandler")
public void httpJobHandler() throws Exception {
// param parse
String param = XxlJobHelper.getJobParam();
if (param==null || param.trim().length()==0) {
XxlJobHelper.log("param["+ param +"] invalid.");
XxlJobHelper.handleFail();
return;
}
String[] httpParams = param.split("\n");
String url = null;
String method = null;
String data = null;
for (String httpParam: httpParams) {
if (httpParam.startsWith("url:")) {
url = httpParam.substring(httpParam.indexOf("url:") + 4).trim();
}
if (httpParam.startsWith("method:")) {
method = httpParam.substring(httpParam.indexOf("method:") + 7).trim().toUpperCase();
}
if (httpParam.startsWith("data:")) {
data = httpParam.substring(httpParam.indexOf("data:") + 5).trim();
}
}
// param valid
if (url==null || url.trim().length()==0) {
XxlJobHelper.log("url["+ url +"] invalid.");
XxlJobHelper.handleFail();
return;
}
if (method==null || !Arrays.asList("GET", "POST").contains(method)) {
XxlJobHelper.log("method["+ method +"] invalid.");
XxlJobHelper.handleFail();
return;
}
boolean isPostMethod = method.equals("POST");
// request
HttpURLConnection connection = null;
BufferedReader bufferedReader = null;
try {
// connection
URL realUrl = new URL(url);
connection = (HttpURLConnection) realUrl.openConnection();
// connection setting
connection.setRequestMethod(method);
connection.setDoOutput(isPostMethod);
connection.setDoInput(true);
connection.setUseCaches(false);
connection.setReadTimeout(5 * 1000);
connection.setConnectTimeout(3 * 1000);
connection.setRequestProperty("connection", "Keep-Alive");
connection.setRequestProperty("Content-Type", "application/json;charset=UTF-8");
connection.setRequestProperty("Accept-Charset", "application/json;charset=UTF-8");
// do connection
connection.connect();
// data
if (isPostMethod && data!=null && data.trim().length()>0) {
DataOutputStream dataOutputStream = new DataOutputStream(connection.getOutputStream());
dataOutputStream.write(data.getBytes("UTF-8"));
dataOutputStream.flush();
dataOutputStream.close();
}
// valid StatusCode
int statusCode = connection.getResponseCode();
if (statusCode != 200) {
throw new RuntimeException("Http Request StatusCode(" + statusCode + ") Invalid.");
}
// result
bufferedReader = new BufferedReader(new InputStreamReader(connection.getInputStream(), "UTF-8"));
StringBuilder result = new StringBuilder();
String line;
while ((line = bufferedReader.readLine()) != null) {
result.append(line);
}
String responseMsg = result.toString();
XxlJobHelper.log(responseMsg);
return;
} catch (Exception e) {
XxlJobHelper.log(e);
XxlJobHelper.handleFail();
return;
} finally {
try {
if (bufferedReader != null) {
bufferedReader.close();
}
if (connection != null) {
connection.disconnect();
}
} catch (Exception e2) {
XxlJobHelper.log(e2);
}
}
}
/**
* 5、生命周期任務(wù)示例：任務(wù)初始化與銷毀時，支持自定義相關(guān)邏輯；
*/
@XxlJob(value = "demoJobHandler2", init = "init", destroy = "destroy")
public void demoJobHandler2() throws Exception {
XxlJobHelper.log("XXL-JOB, Hello World.");
}
public void init(){
logger.info("init");
}
public void destroy(){
logger.info("destory");
}
}
5.啟動執(zhí)行器，此時控制臺上還沒有顯示你啟動的執(zhí)行器，此時需要手動添加，執(zhí)行器的名稱和項目配置中的一致，機器地址可以采用自動注冊的方式也可以使用手動注入的方式
image.png
執(zhí)行器啟動成功后，因為我設(shè)置的是自動注冊，所以成功后會自動注冊機器地址：
image.png
6.新建任務(wù)，然后執(zhí)行測試
image.png
點擊操作下的執(zhí)行一次或者啟動來啟動一個任務(wù)
image.png
7.查看執(zhí)行任務(wù)日志
image.png
完成以上的任務(wù)，我們的xxl-job也就入門了,后面就可以進行集群搭建了
