改寫(xiě)引擎的職責(zé)定位是進(jìn)行SQL的修改，因?yàn)镾hardingSphere的核心目標(biāo)就是屏蔽分庫(kù)分表對(duì)用戶(hù)的影響（當(dāng)然后來(lái)還增加影子表、加解密等功能），使開(kāi)發(fā)者可以按照像原來(lái)傳統(tǒng)單庫(kù)單表一樣編寫(xiě)SQL。表拆分后，表名往往會(huì)帶有編號(hào)或者日期等標(biāo)識(shí)，但應(yīng)用中的SQL中表名并不會(huì)帶有這些標(biāo)識(shí)，一般稱(chēng)之為邏輯表（和未拆分前表名完全相同），因此改寫(xiě)引擎需要用路由引擎計(jì)算得到的真正物理表名替換SQL中的邏輯表名，這樣SQL才能正確執(zhí)行。
除了sharding功能中表名替換，目前在ShardingSphere中需要很多種情況會(huì)進(jìn)行SQL改寫(xiě)，具體有：
- 數(shù)據(jù)分片功能中表名改寫(xiě)；
- 數(shù)據(jù)分片功能中聚合函數(shù)distinct；
- 數(shù)據(jù)分片功能中avg聚合函數(shù)需添加count、sum；
- 數(shù)據(jù)分片功能中索引重命名；
- 數(shù)據(jù)分片功能中分頁(yè)時(shí)offset、rowcount改寫(xiě)；
- 配置分布式自增鍵時(shí)自增列、值添加；
- 加解密功能下對(duì)列、值得添加修改；
- 影子表功能下對(duì)列與值的修改。
代碼調(diào)用分析
回到BasePrepareEngine類(lèi)中，可以看到在使用改寫(xiě)功能前注冊(cè)了改寫(xiě)裝飾器。
org.apache.shardingsphere.underlying.pluggble.prepare.BasePrepareEngine
private Collection<ExecutionUnit> executeRewrite(final String sql, final List<Object> parameters, final RouteContext routeContext) {
registerRewriteDecorator();
//創(chuàng)建SQL改寫(xiě)上下文，主要是生成對(duì)應(yīng)的Token以及參數(shù)
SQLRewriteContext sqlRewriteContext = rewriter.createSQLRewriteContext(sql, parameters, routeContext.getSqlStatementContext(), routeContext);
return routeContext.getRouteResult().getRouteUnits().isEmpty() ? rewrite(sqlRewriteContext) : rewrite(routeContext, sqlRewriteContext);
}
private void registerRewriteDecorator() {
for (Class<? extends SQLRewriteContextDecorator> each : OrderedRegistry.getRegisteredClasses(SQLRewriteContextDecorator.class)) {
SQLRewriteContextDecorator rewriteContextDecorator = createRewriteDecorator(each);
Class<?> ruleClass = (Class<?>) rewriteContextDecorator.getType();
// FIXME rule.getClass().getSuperclass() == ruleClass for orchestration, should decouple extend between orchestration rule and sharding rule
rules.stream().filter(rule -> rule.getClass() == ruleClass || rule.getClass().getSuperclass() == ruleClass).collect(Collectors.toList())
.forEach(rule -> rewriter.registerDecorator(rule, rewriteContextDecorator));
}
}
之后便通過(guò)SQL改寫(xiě)入口類(lèi)SQLRewriteEntry創(chuàng)建SQL改寫(xiě)上下文對(duì)象
/**
* SQL rewrite entry.
*/
@RequiredArgsConstructor
public final class SQLRewriteEntry {
private final SchemaMetaData schemaMetaData;
private final ConfigurationProperties properties;
private final Map<BaseRule, SQLRewriteContextDecorator> decorators = new LinkedHashMap<>();
/**
* Register route decorator.
*
* @param rule rule
* @param decorator SQL rewrite context decorator
*/
public void registerDecorator(final BaseRule rule, final SQLRewriteContextDecorator decorator) {
decorators.put(rule, decorator);
}
/**
* Create SQL rewrite context.
*
* @param sql SQL
* @param parameters parameters
* @param sqlStatementContext SQL statement context
* @param routeContext route context
* @return SQL rewrite context
*/
public SQLRewriteContext createSQLRewriteContext(final String sql, final List<Object> parameters, final SQLStatementContext sqlStatementContext, final RouteContext routeContext) {
SQLRewriteContext result = new SQLRewriteContext(schemaMetaData, sqlStatementContext, sql, parameters);// 創(chuàng)建一個(gè)初始SQL改寫(xiě)上下文
decorate(decorators, result, routeContext);// 進(jìn)行裝飾器處理，其實(shí)就是根據(jù)Statement上下文，生成一系列的Token生成器
result.generateSQLTokens();// 運(yùn)行各Token生成器，解構(gòu)出對(duì)應(yīng)的Token
return result;
}
@SuppressWarnings("unchecked")
private void decorate(final Map<BaseRule, SQLRewriteContextDecorator> decorators, final SQLRewriteContext sqlRewriteContext, final RouteContext routeContext) {
for (Entry<BaseRule, SQLRewriteContextDecorator> entry : decorators.entrySet()) {
BaseRule rule = entry.getKey();
SQLRewriteContextDecorator decorator = entry.getValue();
if (decorator instanceof RouteContextAware) {
((RouteContextAware) decorator).setRouteContext(routeContext);
}
decorator.decorate(rule, properties, sqlRewriteContext);
}
}
}
以最用到的sharding功能實(shí)現(xiàn)的SQL改寫(xiě)上下文裝飾器，看下其實(shí)現(xiàn)邏輯：
org.apache.shardingsphere.sharding.rewrite.context.ShardingSQLRewriteContextDecorator
/**
* SQL rewrite context decorator for sharding.
*/
@Setter
public final class ShardingSQLRewriteContextDecorator implements SQLRewriteContextDecorator<ShardingRule>, RouteContextAware {
private RouteContext routeContext;
@SuppressWarnings("unchecked")
@Override
public void decorate(final ShardingRule shardingRule, final ConfigurationProperties properties, final SQLRewriteContext sqlRewriteContext) {
// 獲取參數(shù)改寫(xiě)器（參數(shù)化SQL才需要），然后依次對(duì)SQL改寫(xiě)上下文中的參數(shù)構(gòu)造器parameterBuilder進(jìn)行改寫(xiě)操作，分片功能下主要是自增鍵以及分頁(yè)參數(shù)
for (ParameterRewriter each : new ShardingParameterRewriterBuilder(shardingRule, routeContext).getParameterRewriters(sqlRewriteContext.getSchemaMetaData())) {
if (!sqlRewriteContext.getParameters().isEmpty() && each.isNeedRewrite(sqlRewriteContext.getSqlStatementContext())) {
each.rewrite(sqlRewriteContext.getParameterBuilder(), sqlRewriteContext.getSqlStatementContext(), sqlRewriteContext.getParameters());
}
}
//添加分片功能下對(duì)應(yīng)的Token生成器
sqlRewriteContext.addSQLTokenGenerators(new ShardingTokenGenerateBuilder(shardingRule, routeContext).getSQLTokenGenerators());
}
…
}
可以看到首先會(huì)通過(guò)ShardingParameterRewriterBuilder創(chuàng)建了數(shù)據(jù)分片功能對(duì)應(yīng)的參數(shù)改寫(xiě)器，包括了insert自增分布式主鍵參數(shù)和分頁(yè)參數(shù)兩個(gè)重寫(xiě)器。
org.apache.shardingsphere.sharding.rewrite.parameter.ShardingParameterRewriterBuilder
/**
* Parameter rewriter builder for sharding.
*/
@RequiredArgsConstructor
public final class ShardingParameterRewriterBuilder implements ParameterRewriterBuilder {
private final ShardingRule shardingRule;
private final RouteContext routeContext;
@Override
public Collection<ParameterRewriter> getParameterRewriters(final SchemaMetaData schemaMetaData) {
Collection<ParameterRewriter> result = getParameterRewriters();
for (ParameterRewriter each : result) {
setUpParameterRewriters(each, schemaMetaData);
}
return result;
}
private static Collection<ParameterRewriter> getParameterRewriters() {
Collection<ParameterRewriter> result = new LinkedList<>();
result.add(new ShardingGeneratedKeyInsertValueParameterRewriter());
result.add(new ShardingPaginationParameterRewriter());
return result;
}
…
}
然后通過(guò)ShardingTokenGenerateBuilder生成數(shù)據(jù)分片Token生成器
org.apache.shardingsphere.sharding.rewrite.token.pojo.ShardingTokenGenerateBuilder
/**
* SQL token generator builder for sharding.
*/
@RequiredArgsConstructor
public final class ShardingTokenGenerateBuilder implements SQLTokenGeneratorBuilder {
private final ShardingRule shardingRule;
private final RouteContext routeContext;
@Override
public Collection<SQLTokenGenerator> getSQLTokenGenerators() {
Collection<SQLTokenGenerator> result = buildSQLTokenGenerators();
for (SQLTokenGenerator each : result) {
if (each instanceof ShardingRuleAware) {
((ShardingRuleAware) each).setShardingRule(shardingRule);
}
if (each instanceof RouteContextAware) {
((RouteContextAware) each).setRouteContext(routeContext);
}
}
return result;
}
private Collection<SQLTokenGenerator> buildSQLTokenGenerators() {
Collection<SQLTokenGenerator> result = new LinkedList<>();
addSQLTokenGenerator(result, new TableTokenGenerator());// 表名token處理，用于真實(shí)表名替換
addSQLTokenGenerator(result, new DistinctProjectionPrefixTokenGenerator());// select distinct關(guān)鍵字處理
addSQLTokenGenerator(result, new ProjectionsTokenGenerator());// select列名處理，主要是衍生列avg處理
addSQLTokenGenerator(result, new OrderByTokenGenerator());// Order by Token處理
addSQLTokenGenerator(result, new AggregationDistinctTokenGenerator());// 聚合函數(shù)的distinct關(guān)鍵字處理
addSQLTokenGenerator(result, new IndexTokenGenerator());// 索引重命名
addSQLTokenGenerator(result, new OffsetTokenGenerator());// offset 改寫(xiě)
addSQLTokenGenerator(result, new RowCountTokenGenerator());// rowCount改寫(xiě)
addSQLTokenGenerator(result, new GeneratedKeyInsertColumnTokenGenerator());// 分布式主鍵列添加，在insert sql列最后添加
addSQLTokenGenerator(result, new GeneratedKeyForUseDefaultInsertColumnsTokenGenerator());// insert SQL使用默認(rèn)列名時(shí)需要完成補(bǔ)齊真實(shí)列名，包括自增列
addSQLTokenGenerator(result, new GeneratedKeyAssignmentTokenGenerator());// SET自增鍵生成
addSQLTokenGenerator(result, new ShardingInsertValuesTokenGenerator());// insert SQL 的values Token解析，為后續(xù)添加自增值做準(zhǔn)備
addSQLTokenGenerator(result, new GeneratedKeyInsertValuesTokenGenerator());//為insert values添加自增列值
return result;
}
private void addSQLTokenGenerator(final Collection<SQLTokenGenerator> sqlTokenGenerators, final SQLTokenGenerator toBeAddedSQLTokenGenerator) {
if (toBeAddedSQLTokenGenerator instanceof IgnoreForSingleRoute && routeContext.getRouteResult().isSingleRouting()) {
return;
}
sqlTokenGenerators.add(toBeAddedSQLTokenGenerator);
}
}
可以看到ShardingTokenGenerateBuilder類(lèi)針對(duì)數(shù)據(jù)分片需要改寫(xiě)SQL的各種情況分別添加了對(duì)應(yīng)的Token生成器，看下最主要的表名Token生成器類(lèi)TableTokenGenerator
org.apache.shardingsphere.sharding.rewrite.token.generator.impl.TableTokenGenerator
/**
* Table token generator.
*/
@Setter
public final class TableTokenGenerator implements CollectionSQLTokenGenerator, ShardingRuleAware {
private ShardingRule shardingRule;
@Override
public boolean isGenerateSQLToken(final SQLStatementContext sqlStatementContext) {
return true;
}
@Override
public Collection<TableToken> generateSQLTokens(final SQLStatementContext sqlStatementContext) {
return sqlStatementContext instanceof TableAvailable ? generateSQLTokens((TableAvailable) sqlStatementContext) : Collections.emptyList();
}
private Collection<TableToken> generateSQLTokens(final TableAvailable sqlStatementContext) {
Collection<TableToken> result = new LinkedList<>();
for (SimpleTableSegment each : sqlStatementContext.getAllTables()) {
if (shardingRule.findTableRule(each.getTableName().getIdentifier().getValue()).isPresent()) {// 分片功能下，添加TableToken
result.add(new TableToken(each.getStartIndex(), each.getStopIndex(), each.getTableName().getIdentifier(), (SQLStatementContext) sqlStatementContext, shardingRule));
}
}
return result;
}
}
可以看到generateSQLTokens方法中，在判斷時(shí)數(shù)據(jù)分片規(guī)則中配置的表后，創(chuàng)建TableToken對(duì)象添加到集合中返回。
org.apache.shardingsphere.sharding.rewrite.token.pojo.TableToken
/**
* Table token.
*/
public final class TableToken extends SQLToken implements Substitutable, RouteUnitAware {
…
@Override
public String toString(final RouteUnit routeUnit) {
String actualTableName = getLogicAndActualTables(routeUnit).get(identifier.getValue().toLowerCase());
actualTableName = null == actualTableName ? identifier.getValue().toLowerCase() : actualTableName;
//替換成真實(shí)物理表名
return Joiner.on("").join(identifier.getQuoteCharacter().getStartDelimiter(), actualTableName, identifier.getQuoteCharacter().getEndDelimiter());
}
…
}
TableToken的toString方法即根據(jù)RouteUnit對(duì)象生成該Token在SQL改寫(xiě)時(shí)需要替換成的字符串。
再次回到BasePrepareEngine類(lèi)
org.apache.shardingsphere.underlying.pluggble.prepare.BasePrepareEngine# executeRewrite方法，在創(chuàng)建完SQL改寫(xiě)上下文后，調(diào)用了rewrite方法生成執(zhí)行單元集合
private Collection<ExecutionUnit> executeRewrite(final String sql, final List<Object> parameters, final RouteContext routeContext) {
registerRewriteDecorator();
//創(chuàng)建SQL重寫(xiě)上下文，主要是生成對(duì)應(yīng)的Token以及參數(shù)
SQLRewriteContext sqlRewriteContext = rewriter.createSQLRewriteContext(sql, parameters, routeContext.getSqlStatementContext(), routeContext);
return routeContext.getRouteResult().getRouteUnits().isEmpty() ? rewrite(sqlRewriteContext) : rewrite(routeContext, sqlRewriteContext);
}
可以看到根據(jù)路由單元數(shù)量，分別對(duì)應(yīng)兩個(gè)rewrite私有方法
// 此方法負(fù)責(zé)將SQL改寫(xiě)上下文轉(zhuǎn)化為執(zhí)行單元ExecutionUnit集合
private Collection<ExecutionUnit> rewrite(final SQLRewriteContext sqlRewriteContext) {
SQLRewriteResult sqlRewriteResult = new SQLRewriteEngine().rewrite(sqlRewriteContext);// 將SQL改寫(xiě)上下文轉(zhuǎn)化為SQL改寫(xiě)結(jié)果，主要是獲取改寫(xiě)后的SQL與參數(shù)
String dataSourceName = metaData.getDataSources().getAllInstanceDataSourceNames().iterator().next();
return Collections.singletonList(new ExecutionUnit(dataSourceName, new SQLUnit(sqlRewriteResult.getSql(), sqlRewriteResult.getParameters())));
}
private Collection<ExecutionUnit> rewrite(final RouteContext routeContext, final SQLRewriteContext sqlRewriteContext) {
Collection<ExecutionUnit> result = new LinkedHashSet<>();
for (Entry<RouteUnit, SQLRewriteResult> entry : new SQLRouteRewriteEngine().rewrite(sqlRewriteContext, routeContext.getRouteResult()).entrySet()) {
result.add(new ExecutionUnit(entry.getKey().getDataSourceMapper().getActualName(), new SQLUnit(entry.getValue().getSql(), entry.getValue().getParameters())));
}
return result;
}
第一個(gè)rewrite方法中，創(chuàng)建SQLRewriteEngine實(shí)例，然后執(zhí)行其rewrite方法生成SQLRewriteResult對(duì)象，通過(guò)此對(duì)象獲取改寫(xiě)后的SQL以及參數(shù)，之后創(chuàng)建SQLUnit對(duì)象、創(chuàng)建ExecutionUnit對(duì)象，添加到集合中返回。接下來(lái)看下SQLRewriteEngine類(lèi)：
org.apache.shardingsphere.underlying.rewrite.engine.SQLRewriteEngine
/**
* SQL rewrite engine.
*/
public final class SQLRewriteEngine {
/**
* Rewrite SQL and parameters.
*
* @param sqlRewriteContext SQL rewrite context
* @return SQL rewrite result
*/
public SQLRewriteResult rewrite(final SQLRewriteContext sqlRewriteContext) {
// 將SQL改寫(xiě)上下文中Token轉(zhuǎn)化成SQL， 獲取改寫(xiě)后參數(shù)，然后構(gòu)造成SQL改寫(xiě)結(jié)果返回
return new SQLRewriteResult(new DefaultSQLBuilder(sqlRewriteContext).toSQL(), sqlRewriteContext.getParameterBuilder().getParameters());
}
}
可以看到該類(lèi)通過(guò)DefaultSQLBuilder類(lèi)對(duì)象的toSQL()生成了改寫(xiě)后SQL，然后創(chuàng)建了SQLRewriteResult實(shí)例，那么接下來(lái)繼續(xù)看下DefaultSQLBuilder類(lèi)
org.apache.shardingsphere.underlying.rewrite.sql.impl.DefaultSQLBuilder
/**
* Default SQL builder.
*/
public final class DefaultSQLBuilder extends AbstractSQLBuilder {
public DefaultSQLBuilder(final SQLRewriteContext context) {
super(context);
}
@Override
protected String getSQLTokenText(final SQLToken sqlToken) {
return sqlToken.toString();// 返回Token對(duì)應(yīng)的文本字符
}
}
DefaultSQLBuilder類(lèi)只是實(shí)現(xiàn)了getSQLTokenText方法，調(diào)用Token.toString方法返回。
看下其父類(lèi)AbstractSQLBuilder
org.apache.shardingsphere.underlying.rewrite.sql.impl.AbstractSQLBuilder
public abstract class AbstractSQLBuilder implements SQLBuilder {
private final SQLRewriteContext context;
@Override
public final String toSQL() {
if (context.getSqlTokens().isEmpty()) {
return context.getSql();
}
Collections.sort(context.getSqlTokens());// 按照Token的起始位置排序
StringBuilder result = new StringBuilder();
result.append(context.getSql().substring(0, context.getSqlTokens().get(0).getStartIndex()));// 添加第一個(gè)Token之前的原始SQL
for (SQLToken each : context.getSqlTokens()) {
result.append(getSQLTokenText(each));// 添加Token對(duì)應(yīng)的SQL片段
result.append(getConjunctionText(each));// 添加Token之間的連接字符
}
return result.toString();
}
…
}
可以看到在A(yíng)bstractSQLBuilder的toSQL方法中，對(duì)Token進(jìn)行排序，然后通過(guò)拼接原始SQL和替換的Token以及連接符，最后形成完整的改寫(xiě)SQL。
回頭再看下BasePrepareEngine類(lèi)中的第二個(gè)rewrite方法
private Collection<ExecutionUnit> rewrite(final RouteContext routeContext, final SQLRewriteContext sqlRewriteContext) {
Collection<ExecutionUnit> result = new LinkedHashSet<>();
for (Entry<RouteUnit, SQLRewriteResult> entry : new SQLRouteRewriteEngine().rewrite(sqlRewriteContext, routeContext.getRouteResult()).entrySet()) {
result.add(new ExecutionUnit(entry.getKey().getDataSourceMapper().getActualName(), new SQLUnit(entry.getValue().getSql(), entry.getValue().getParameters())));
}
return result;
}
可以看到通過(guò)執(zhí)行了SQLRouteRewriteEngine對(duì)象的rewrite方法返回了一個(gè)Map<RouteUnit, SQLRewriteResult>對(duì)象，然后遍歷構(gòu)建了ExecutionUnit，然后添加到集合中進(jìn)行返回。那么我們看下SQLRouteRewriteEngine類(lèi)：
org.apache.shardingsphere.underlying.rewrite.engine.SQLRouteRewriteEngine
/**
* SQL rewrite engine with route.
*/
public final class SQLRouteRewriteEngine {
/**
* Rewrite SQL and parameters.
*
* @param sqlRewriteContext SQL rewrite context
* @param routeResult route result
* @return SQL map of route unit and rewrite result
*/
public Map<RouteUnit, SQLRewriteResult> rewrite(final SQLRewriteContext sqlRewriteContext, final RouteResult routeResult) {
Map<RouteUnit, SQLRewriteResult> result = new LinkedHashMap<>(routeResult.getRouteUnits().size(), 1);
for (RouteUnit each : routeResult.getRouteUnits()) {
result.put(each, new SQLRewriteResult(new RouteSQLBuilder(sqlRewriteContext, each).toSQL(), getParameters(sqlRewriteContext.getParameterBuilder(), routeResult, each)));
}
return result;
}
…
}
可以看到，與SQLRewriteEngine類(lèi)相似，只不過(guò)改為遍歷路由結(jié)果中包含的RouteUnit，然后分別創(chuàng)建對(duì)應(yīng)的SQLRewriteResult實(shí)例，這里構(gòu)建改寫(xiě)SQL的類(lèi)用了一個(gè)RouteSQLBuilder類(lèi)
org.apache.shardingsphere.underlying.rewrite.sql.impl.RouteSQLBuilder
/**
* SQL builder with route.
*/
public final class RouteSQLBuilder extends AbstractSQLBuilder {
private final RouteUnit routeUnit;
public RouteSQLBuilder(final SQLRewriteContext context, final RouteUnit routeUnit) {
super(context);
this.routeUnit = routeUnit;
}
@Override
protected String getSQLTokenText(final SQLToken sqlToken) {
if (sqlToken instanceof RouteUnitAware) {
return ((RouteUnitAware) sqlToken).toString(routeUnit);
}
return sqlToken.toString();
}
}
…
}
這個(gè)類(lèi)與DefaultSQLBuilder類(lèi)似，也繼承自AbstractSQLBuilder類(lèi)，只不過(guò)getSQLTokenText方法會(huì)判斷是否是RouteUnitAware類(lèi)型的Token，如果是則調(diào)用RouteUnit參數(shù)的toSQL方法生成SQL。
總結(jié)
最后總結(jié)下改寫(xiě)引擎的執(zhí)行流程：
改寫(xiě)引擎的輸入即為路由上下文RouteContext，輸出為SQL改寫(xiě)上下文SQLRewriteResult。
- 改寫(xiě)功能入口SQLRewriteEntry類(lèi)中創(chuàng)建一個(gè)初始化SQLRewriteResult對(duì)象。
- 順序執(zhí)行已注冊(cè)的SQL改寫(xiě)上下文裝飾器，添加對(duì)應(yīng)的一系列Token生成器對(duì)象。
- 調(diào)用SQLRewriteContext. generateSQLTokens方法，運(yùn)行各Token生成器，解析出對(duì)應(yīng)的Token。
- 由SQL改寫(xiě)引擎SQLRewriteEngine或者SQLRouteRewriteEngine類(lèi)調(diào)用DefaultSQLBuilder或RouteSQLBuilder類(lèi)將解析出的Token拼接成改寫(xiě)后的SQL，基于此SQL和參數(shù)，創(chuàng)建SQLRewriteResult實(shí)例返回。
- 基于SQLRewriteResult，創(chuàng)建ExecutionUnit，然后再封裝成ExecutionContext。
改寫(xiě)引擎執(zhí)行流程
