final class ExecutorCallAdapterFactory extends CallAdapter.Factory {
final Executor callbackExecutor;
ExecutorCallAdapterFactory(Executor callbackExecutor) {
this.callbackExecutor = callbackExecutor;
}
@Override public @Nullable CallAdapter, ?> get(
Type returnType, Annotation[] annotations, Retrofit retrofit) {
// returnType在文章之前分析过 是Call ,getRawType返回是Call
if (getRawType(returnType) != Call.class) {
return null;
}
//这里根据returnType本文返回为Call ,那么responseType就是DataModel
final Type responseType = Utils.getCallResponseType(returnType);
return new CallAdapter
传入的参数是Call,以下相当于通过反射获取泛型的参数DataModel:
static Type getCallResponseType(Type returnType) {
if (!(returnType instanceof ParameterizedType)) {
throw new IllegalArgumentException(
"Call return type must be parameterized as Call or Call extends Foo>");
}
return getParameterUpperBound(0, (ParameterizedType) returnType);
}
static Type getParameterUpperBound(int index, ParameterizedType type) {
Type[] types = type.getActualTypeArguments();
if (index < 0 || index >= types.length) {
throw new IllegalArgumentException(
"Index " + index + " not in range [0," + types.length + ") for " + type);
}
Type paramType = types[index];
if (paramType instanceof WildcardType) {
return ((WildcardType) paramType).getUpperBounds()[0];
}
return paramType;
}
public final class GsonConverterFactory extends Converter.Factory {
/**
* Create an instance using a default {@link Gson} instance for conversion. Encoding to JSON and
* decoding from JSON (when no charset is specified by a header) will use UTF-8.
*/
public static GsonConverterFactory create() {
return create(new Gson());
}
/**
* Create an instance using {@code gson} for conversion. Encoding to JSON and
* decoding from JSON (when no charset is specified by a header) will use UTF-8.
*/
@SuppressWarnings("ConstantConditions") // Guarding public API nullability.
public static GsonConverterFactory create(Gson gson) {
if (gson == null) throw new NullPointerException("gson == null");
return new GsonConverterFactory(gson);
}
private final Gson gson;
private GsonConverterFactory(Gson gson) {
this.gson = gson;
}
...
}
GsonConverterFactory就是对Gson的封装处理,数据格式的适配函数如下:
@Override
public Converter responseBodyConverter(Type type, Annotation[] annotations,
Retrofit retrofit) {
TypeAdapter> adapter = gson.getAdapter(TypeToken.get(type));
return new GsonResponseBodyConverter<>(gson, adapter);
}
public TypeAdapter getAdapter(TypeToken type) {
TypeAdapter> cached = typeTokenCache.get(type == null ? NULL_KEY_SURROGATE : type);
if (cached != null) {
return (TypeAdapter) cached;
}
Map, FutureTypeAdapter>> threadCalls = calls.get();
boolean requiresThreadLocalCleanup = false;
if (threadCalls == null) {
threadCalls = new HashMap, FutureTypeAdapter>>();
calls.set(threadCalls);
requiresThreadLocalCleanup = true;
}
// the key and value type parameters always agree
FutureTypeAdapter ongoingCall = (FutureTypeAdapter) threadCalls.get(type);
if (ongoingCall != null) {
return ongoingCall;
}
try {
FutureTypeAdapter call = new FutureTypeAdapter();
threadCalls.put(type, call);
for (TypeAdapterFactory factory : factories) {
TypeAdapter candidate = factory.create(this, type);
if (candidate != null) {
call.setDelegate(candidate);
typeTokenCache.put(type, candidate);
return candidate;
}
}
throw new IllegalArgumentException("GSON cannot handle " + type);
} finally {
threadCalls.remove(type);
if (requiresThreadLocalCleanup) {
calls.remove();
}
}
}
public static TypeToken> get(Type type) {
return new TypeToken(type);
}
TypeToken(Type type) {
//这里传入的参数是Call泛型的实际类型,本文是DataModel
this.type = $Gson$Types.canonicalize($Gson$Preconditions.checkNotNull(type));
this.rawType = (Class super T>) $Gson$Types.getRawType(this.type);
this.hashCode = this.type.hashCode();
}
public static Type canonicalize(Type type) {
// 很显然,本文可以通过这个TypeAdapter来转完成数据适配,这里type在本文中是DataModel
if (type instanceof Class) {//原始类型
Class> c = (Class>) type;
return c.isArray() ? new GenericArrayTypeImpl(canonicalize(c.getComponentType())) : c;
} else if (type instanceof ParameterizedType) {//参数化类型
ParameterizedType p = (ParameterizedType) type;
return new ParameterizedTypeImpl(p.getOwnerType(),
p.getRawType(), p.getActualTypeArguments());
} else if (type instanceof GenericArrayType) {//数组类型
GenericArrayType g = (GenericArrayType) type;
return new GenericArrayTypeImpl(g.getGenericComponentType());
} else if (type instanceof WildcardType) {//通配符类型
WildcardType w = (WildcardType) type;
return new WildcardTypeImpl(w.getUpperBounds(), w.getLowerBounds());
} else {
// type is either serializable as-is or unsupported
return type;
}
}
public static Class> getRawType(Type type) {
// 很显然,本文直接返回DataModel
if (type instanceof Class>) {
// type is a normal class.
return (Class>) type;
} else if (type instanceof ParameterizedType) {
ParameterizedType parameterizedType = (ParameterizedType) type;
// I'm not exactly sure why getRawType() returns Type instead of Class.
// Neal isn't either but suspects some pathological case related
// to nested classes exists.
Type rawType = parameterizedType.getRawType();
checkArgument(rawType instanceof Class);
return (Class>) rawType;
} else if (type instanceof GenericArrayType) {
Type componentType = ((GenericArrayType)type).getGenericComponentType();
return Array.newInstance(getRawType(componentType), 0).getClass();
} else if (type instanceof TypeVariable) {
// we could use the variable's bounds, but that won't work if there are multiple.
// having a raw type that's more general than necessary is okay
return Object.class;
} else if (type instanceof WildcardType) {
return getRawType(((WildcardType) type).getUpperBounds()[0]);
} else {
String className = type == null ? "null" : type.getClass().getName();
throw new IllegalArgumentException("Expected a Class, ParameterizedType, or "
+ "GenericArrayType, but <" + type + "> is of type " + className);
}
}
在Gson初始化时添加了各种TypeAdapters:
Gson(final Excluder excluder, final FieldNamingStrategy fieldNamingStrategy,
final Map> instanceCreators, boolean serializeNulls,
boolean complexMapKeySerialization, boolean generateNonExecutableGson, boolean htmlSafe,
boolean prettyPrinting, boolean lenient, boolean serializeSpecialFloatingPointValues,
LongSerializationPolicy longSerializationPolicy,
List typeAdapterFactories) {
this.constructorConstructor = new ConstructorConstructor(instanceCreators);
this.excluder = excluder;
this.fieldNamingStrategy = fieldNamingStrategy;
this.serializeNulls = serializeNulls;
this.generateNonExecutableJson = generateNonExecutableGson;
this.htmlSafe = htmlSafe;
this.prettyPrinting = prettyPrinting;
this.lenient = lenient;
List factories = new ArrayList();
// built-in type adapters that cannot be overridden
factories.add(TypeAdapters.JSON_ELEMENT_FACTORY);
factories.add(ObjectTypeAdapter.FACTORY);
// the excluder must precede all adapters that handle user-defined types
factories.add(excluder);
// user's type adapters
factories.addAll(typeAdapterFactories);
// type adapters for basic platform types
factories.add(TypeAdapters.STRING_FACTORY);
factories.add(TypeAdapters.INTEGER_FACTORY);
factories.add(TypeAdapters.BOOLEAN_FACTORY);
factories.add(TypeAdapters.BYTE_FACTORY);
factories.add(TypeAdapters.SHORT_FACTORY);
TypeAdapter longAdapter = longAdapter(longSerializationPolicy);
factories.add(TypeAdapters.newFactory(long.class, Long.class, longAdapter));
factories.add(TypeAdapters.newFactory(double.class, Double.class,
doubleAdapter(serializeSpecialFloatingPointValues)));
factories.add(TypeAdapters.newFactory(float.class, Float.class,
floatAdapter(serializeSpecialFloatingPointValues)));
factories.add(TypeAdapters.NUMBER_FACTORY);
factories.add(TypeAdapters.ATOMIC_INTEGER_FACTORY);
factories.add(TypeAdapters.ATOMIC_BOOLEAN_FACTORY);
factories.add(TypeAdapters.newFactory(AtomicLong.class, atomicLongAdapter(longAdapter)));
factories.add(TypeAdapters.newFactory(AtomicLongArray.class, atomicLongArrayAdapter(longAdapter)));
factories.add(TypeAdapters.ATOMIC_INTEGER_ARRAY_FACTORY);
factories.add(TypeAdapters.CHARACTER_FACTORY);
factories.add(TypeAdapters.STRING_BUILDER_FACTORY);
factories.add(TypeAdapters.STRING_BUFFER_FACTORY);
factories.add(TypeAdapters.newFactory(BigDecimal.class, TypeAdapters.BIG_DECIMAL));
factories.add(TypeAdapters.newFactory(BigInteger.class, TypeAdapters.BIG_INTEGER));
factories.add(TypeAdapters.URL_FACTORY);
factories.add(TypeAdapters.URI_FACTORY);
factories.add(TypeAdapters.UUID_FACTORY);
factories.add(TypeAdapters.CURRENCY_FACTORY);
factories.add(TypeAdapters.LOCALE_FACTORY);
factories.add(TypeAdapters.INET_ADDRESS_FACTORY);
factories.add(TypeAdapters.BIT_SET_FACTORY);
factories.add(DateTypeAdapter.FACTORY);
factories.add(TypeAdapters.CALENDAR_FACTORY);
factories.add(TimeTypeAdapter.FACTORY);
factories.add(SqlDateTypeAdapter.FACTORY);
factories.add(TypeAdapters.TIMESTAMP_FACTORY);
factories.add(ArrayTypeAdapter.FACTORY);
factories.add(TypeAdapters.CLASS_FACTORY);
// type adapters for composite and user-defined types
factories.add(new CollectionTypeAdapterFactory(constructorConstructor));
factories.add(new MapTypeAdapterFactory(constructorConstructor, complexMapKeySerialization));
this.jsonAdapterFactory = new JsonAdapterAnnotationTypeAdapterFactory(constructorConstructor);
factories.add(jsonAdapterFactory);
factories.add(TypeAdapters.ENUM_FACTORY);
factories.add(new ReflectiveTypeAdapterFactory(
constructorConstructor, fieldNamingStrategy, excluder, jsonAdapterFactory));
this.factories = Collections.unmodifiableList(factories);
}
public static final TypeAdapterFactory JSON_ELEMENT_FACTORY
= newTypeHierarchyFactory(JsonElement.class, JSON_ELEMENT);
/**
* Returns a factory for all subtypes of {@code typeAdapter}. We do a runtime check to confirm
* that the deserialized type matches the type requested.
*/
public static TypeAdapterFactory newTypeHierarchyFactory(
final Class clazz, final TypeAdapter typeAdapter) {
return new TypeAdapterFactory() {
@SuppressWarnings("unchecked")
@Override public TypeAdapter create(Gson gson, TypeToken typeToken) {
//从TypeToken源码可知,这里返回的就是实际类型,本文的DataModel
final Class super T2> requestedType = typeToken.getRawType();
if (!clazz.isAssignableFrom(requestedType)) {
return null;
}
return (TypeAdapter) new TypeAdapter() {
@Override public void write(JsonWriter out, T1 value) throws IOException {
typeAdapter.write(out, value);
}
@Override public T1 read(JsonReader in) throws IOException {
T1 result = typeAdapter.read(in);
if (result != null && !requestedType.isInstance(result)) {
throw new JsonSyntaxException("Expected a " + requestedType.getName()
+ " but was " + result.getClass().getName());
}
return result;
}
};
}
@Override public String toString() {
return "Factory[typeHierarchy=" + clazz.getName() + ",adapter=" + typeAdapter + "]";
}
};
}
Response getResponseWithInterceptorChain() throws IOException {
// Build a full stack of interceptors.
List interceptors = new ArrayList<>();
interceptors.addAll(client.interceptors());
interceptors.add(retryAndFollowUpInterceptor);
interceptors.add(new BridgeInterceptor(client.cookieJar()));
interceptors.add(new CacheInterceptor(client.internalCache()));
interceptors.add(new ConnectInterceptor(client));
if (!forWebSocket) {
interceptors.addAll(client.networkInterceptors());
}
interceptors.add(new CallServerInterceptor(forWebSocket));
Interceptor.Chain chain = new RealInterceptorChain(interceptors, null, null, null, 0,
originalRequest, this, eventListener, client.connectTimeoutMillis(),
client.readTimeoutMillis(), client.writeTimeoutMillis());
return chain.proceed(originalRequest);
}
在这里添加了interceptors并且创建了一个RealInterceptorChain:
@Override public Response proceed(Request request) throws IOException {
return proceed(request, streamAllocation, httpCodec, connection);
}
public Response proceed(Request request, StreamAllocation streamAllocation, HttpCodec httpCodec,
RealConnection connection) throws IOException {
if (index >= interceptors.size()) throw new AssertionError();
calls++;
// If we already have a stream, confirm that the incoming request will use it.
if (this.httpCodec != null && !this.connection.supportsUrl(request.url())) {
throw new IllegalStateException("network interceptor " + interceptors.get(index - 1)
+ " must retain the same host and port");
}
// If we already have a stream, confirm that this is the only call to chain.proceed().
if (this.httpCodec != null && calls > 1) {
throw new IllegalStateException("network interceptor " + interceptors.get(index - 1)
+ " must call proceed() exactly once");
}
// Call the next interceptor in the chain.
// index首次是0,然后index + 1实现链式调用
RealInterceptorChain next = new RealInterceptorChain(interceptors, streamAllocation, httpCodec,
connection, index + 1, request, call, eventListener, connectTimeout, readTimeout,
writeTimeout);
// 这里在之前添加过client.interceptors()、BridgeInterceptor等
Interceptor interceptor = interceptors.get(index);
// 通过各种interceptor对数据做处理
Response response = interceptor.intercept(next);
// Confirm that the next interceptor made its required call to chain.proceed().
if (httpCodec != null && index + 1 < interceptors.size() && next.calls != 1) {
throw new IllegalStateException("network interceptor " + interceptor
+ " must call proceed() exactly once");
}
// Confirm that the intercepted response isn't null.
if (response == null) {
throw new NullPointerException("interceptor " + interceptor + " returned null");
}
if (response.body() == null) {
throw new IllegalStateException(
"interceptor " + interceptor + " returned a response with no body");
}
return response;
}
注意client.interceptors()是在最开始就处理的,比如声明的Log信息拦截器:
OkHttpClient.Builder builder = new OkHttpClient.Builder();
OkHttpClient okHttpClient = builder.build();
if (BuildConfig.DEBUG) {
HttpLoggingInterceptor loggingInterceptor = new HttpLoggingInterceptor();
loggingInterceptor.setLevel(HttpLoggingInterceptor.Level.BODY);
builder.addInterceptor(loggingInterceptor);
}
@Override public Response intercept(Chain chain) throws IOException {
RealInterceptorChain realChain = (RealInterceptorChain) chain;
HttpCodec httpCodec = realChain.httpStream();
StreamAllocation streamAllocation = realChain.streamAllocation();
RealConnection connection = (RealConnection) realChain.connection();
Request request = realChain.request();
long sentRequestMillis = System.currentTimeMillis();
realChain.eventListener().requestHeadersStart(realChain.call());
httpCodec.writeRequestHeaders(request);
realChain.eventListener().requestHeadersEnd(realChain.call(), request);
Response.Builder responseBuilder = null;
if (HttpMethod.permitsRequestBody(request.method()) && request.body() != null) {
// If there's a "Expect: 100-continue" header on the request, wait for a "HTTP/1.1 100
// Continue" response before transmitting the request body. If we don't get that, return
// what we did get (such as a 4xx response) without ever transmitting the request body.
if ("100-continue".equalsIgnoreCase(request.header("Expect"))) {
httpCodec.flushRequest();
realChain.eventListener().responseHeadersStart(realChain.call());
responseBuilder = httpCodec.readResponseHeaders(true);
}
if (responseBuilder == null) {
// Write the request body if the "Expect: 100-continue" expectation was met.
realChain.eventListener().requestBodyStart(realChain.call());
long contentLength = request.body().contentLength();
CountingSink requestBodyOut =
new CountingSink(httpCodec.createRequestBody(request, contentLength));
BufferedSink bufferedRequestBody = Okio.buffer(requestBodyOut);
request.body().writeTo(bufferedRequestBody);
bufferedRequestBody.close();
realChain.eventListener()
.requestBodyEnd(realChain.call(), requestBodyOut.successfulCount);
} else if (!connection.isMultiplexed()) {
// If the "Expect: 100-continue" expectation wasn't met, prevent the HTTP/1 connection
// from being reused. Otherwise we're still obligated to transmit the request body to
// leave the connection in a consistent state.
streamAllocation.noNewStreams();
}
}
httpCodec.finishRequest();
if (responseBuilder == null) {
realChain.eventListener().responseHeadersStart(realChain.call());
responseBuilder = httpCodec.readResponseHeaders(false);
}
Response response = responseBuilder
.request(request)
.handshake(streamAllocation.connection().handshake())
.sentRequestAtMillis(sentRequestMillis)
.receivedResponseAtMillis(System.currentTimeMillis())
.build();
int code = response.code();
if (code == 100) {
// server sent a 100-continue even though we did not request one.
// try again to read the actual response
responseBuilder = httpCodec.readResponseHeaders(false);
response = responseBuilder
.request(request)
.handshake(streamAllocation.connection().handshake())
.sentRequestAtMillis(sentRequestMillis)
.receivedResponseAtMillis(System.currentTimeMillis())
.build();
code = response.code();
}
realChain.eventListener()
.responseHeadersEnd(realChain.call(), response);
if (forWebSocket && code == 101) {
// Connection is upgrading, but we need to ensure interceptors see a non-null response body.
response = response.newBuilder()
.body(Util.EMPTY_RESPONSE)
.build();
} else {
response = response.newBuilder()
.body(httpCodec.openResponseBody(response))
.build();
}
if ("close".equalsIgnoreCase(response.request().header("Connection"))
|| "close".equalsIgnoreCase(response.header("Connection"))) {
streamAllocation.noNewStreams();
}
if ((code == 204 || code == 205) && response.body().contentLength() > 0) {
throw new ProtocolException(
"HTTP " + code + " had non-zero Content-Length: " + response.body().contentLength());
}
return response;
}
1. aggregateByKey的运行机制
/**
* Aggregate the values of each key, using given combine functions and a neutral "zero value".
* This function can return a different result type
spark-sql是Spark bin目录下的一个可执行脚本,它的目的是通过这个脚本执行Hive的命令,即原来通过
hive>输入的指令可以通过spark-sql>输入的指令来完成。
spark-sql可以使用内置的Hive metadata-store,也可以使用已经独立安装的Hive的metadata store
关于Hive build into Spark
// Max value in Array
var arr = [1,2,3,5,3,2];Math.max.apply(null, arr); // 5
// Max value in Jaon Array
var arr = [{"x":"8/11/2009","y":0.026572007},{"x"
在使用XMlhttpRequest对象发送请求和响应之前,必须首先使用javaScript对象创建一个XMLHttpRquest对象。
var xmlhttp;
function getXMLHttpRequest(){
if(window.ActiveXObject){
xmlhttp:new ActiveXObject("Microsoft.XMLHTTP