首先咱们了解一下这几个概念:哈希表、时间复杂度、链表
Plenty of Foundation classes are class clusters and NSDictionary
is no exception. For quite a long time NSDictionary
used CFDictionary
as its default implementation, however, starting with iOS 6.0 things have changed:
1 2 |
|
Similarly to __NSArrayM
, __NSDictionaryI
rests within the CoreFoundation framework, in spite of being publicly presented as a part of Foundation. Running the library through class-dump generates the following ivar layout:
1 2 3 4 5 |
|
It’s surprisingly short. There doesn’t seem to be any pointer to either keys or objects storage. As we will soon see, __NSDictionary
literally keeps its storage to itself.
To understand where __NSDictionaryI
keeps its contents, let’s take a quick tour through the instance creation process. There is just one class method that’s responsible for spawning new instances of __NSDictionaryI
. According to class-dump, the method has the following signature:
|
It takes five arguments, of which only the first one is named. Seriously, if you were to use it in a @selector
statement it would have a form of @selector(__new:::::)
. The first three arguments are easily inferred by setting a breakpoint on this method and peeking into the contents of x2
, x3
and x4
registers which contain the array of keys, array of objects and number of keys (objects) respectively. Notice, that keys and objects arrays are swapped in comparison to the public facing API which takes a form of:
|
It doesn’t matter whether an argument is defined as const id *
or const id []
since arrays decay into pointers when passed as function arguments.
With three arguments covered we’re left with the two unidentified boolean parameters. I’ve done some assembly digging with the following results: the fourth argument governs whether the keys should be copied, and the last one decides whether the arguments should not be retained. We can now rewrite the method with named parameters:
|
Unfortunately, we don’t have explicit access to this private method, so by using the regular means of allocation the last two arguments are always set to YES
and NO
respectively. It is nonetheless interesting that __NSDictionaryI
is capable of a more sophisticated keys and objects control.
Skimming through the disassembly of + __new:::::
reveals that both malloc
and calloc
are nowhere to be found. Instead, the method calls into __CFAllocateObject2
passing the __NSDictionaryI
class as first argument and requested storage size as a second. Stepping down into the sea of ARM64 shows that the first thing __CFAllocateObject2
does is call into class_createInstance
with the exact same arguments.
Fortunately, at this point we have access to the source code of Objective-C runtime which makes further investigation much easier.
The class_createInstance(Class cls, size_t extraBytes)
function merely calls into _class_createInstanceFromZone
passing nil
as a zone, but this is the final step of object allocation. While the function itself has many additional checks for different various circumstances, its gist can be covered with just three lines:
|
The extraBytes
argument couldn’t have been more descriptive. It’s literally the number of extra bytes that inflate the default instance size. As an added bonus, notice that it’s the calloc
call that ensures all the ivars are zeroed out when the object gets allocated.
The indexed ivars section is nothing more than an additional space that sits at the end of regular ivars:
Allocating space on its own doesn’t sound very thrilling so the runtime publishes an accessor:
|
There is no magic whatsoever in this function, it just returns a pointer to the beginning of indexed ivars section:
There are few cool things about indexed ivars. First of all, each instance can have different amount of extra bytes dedicated to it. This is exactly the feature __NSDictionaryI
makes use of.
Secondly, they provide faster access to the storage. It all comes down to being cache-friendly. Generally speaking, jumping to random memory locations (by dereferencing a pointer) can be expensive. Since the object has just been accessed (somebody has called a method on it), it’s very likely that its indexed ivars have landed in cache. By keeping everything that’s needed very close, the object can provide as good performance as possible.
Finally, indexed ivars can be used as a crude defensive measure to make object’s internals invisible to the utilities like class-dump. This is a very basic protection since a dedicated attacker can simply look for object_getIndexedIvars
calls in the disassembly or randomly probe the instance past its regular ivars section to see what’s going on.
While powerful, indexed ivars come with two caveats. First of all, class_createInstance
can’t be used under ARC, so you’ll have to compile some parts of your class with -fno-objc-arc
flag to make it shine. Secondly, the runtime doesn’t keep the indexed ivar size information anywhere. Even though dealloc
will clean everything up (as it calls free
internally), you should keep the storage size somewhere, assuming you use variable number of extra bytes.
Although at this point we could poke the __NSDictionaryI
instances to figure out how they work, the ultimate truth lies within the assembly. Instead of going through the entire wall of ARM64 we will discuss the equivalent Objective-C code instead.
The class itself implements very few methods, but I claim the most important is objectForKey:
– this is what we’re going to discuss in more detail. Since I made the assembly analysis anyway, you can read it on a separate page. It’s dense, but the thorough pass should convince you the following code is more or less correct.
Unfortunately, I don’t have access to the Apple’s code base, so the reverse-engineered code below is not identical to the original implementation. On the other hand, it seems to be working well and I’ve yet to find an edge case that behaves differently in comparison to the genuine method.
The following code is written from the perspective of __NSDictionaryI
class:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 |
|
When you take a closer look at the C code you might notice something strange about key fetching. It’s always taken from even offsets, while the returned object is at the very next index. This is the dead giveaway of __NSDictionaryI
’s internal storage: it keeps keys and objects alternately:
Update: Joan Lluch provided a very convincing explanation for this layout. The original code could use an array of very simple structs:
|
The objectForKey:
method is very straightforward and I highly encourage you to walk through it in your head. It’s nonetheless worth pointing out a few things. First of all, the _szidx
ivar is used as an index into the __NSDictionarySizes
array, thus it most likely stands for “size index”.
Secondly, the only method called on the passed key is hash
. The reminder of dividing key’s hash value by dictionary’s size is used to calculate the offset into the index ivars section.
If the key at the offset is nil
, we simply return nil
, the job is done:
However, if the key at the offset is non nil
, then the two cases may occur. If the keys are equal, then we return the adjacent object. If they’re not equal then the hash collision occurred and we have to keep looking further. __NSDictionaryI
simply keeps looking until either match or nil
is found:
This kind of searching is known as linear probing. Notice how __NSDictionaryI
wraps the fetchIndex
around when the storage end is hit. The for
loop is there to limit the number of checks – if the storage was full and the loop condition was missing we’d keep looking forever.
__NSDictionarySizes & __NSDictionaryCapacities
We already know __NSDictionarySizes
is some kind of array that stores different possible sizes of __NSDictionaryI
. We can reason that it’s an array of NSUInteger
s and indeed, if we ask Hopper to treat the values as 64-bit unsigned integers it suddenly makes a lot of sense:
|
In a more familiar decimal form it presents as a beautiful list of 64 primes starting with the following sequence: 0, 3, 7, 13, 23, 41, 71, 127. Notice, that those are not consecutive primes which begs the question: what’s the average ratio of the two neighboring numbers? It’s actually around 1.637
– a very close match to the 1.625
which was the growth factor for NSMutableArray
. For details of why primes are used for the storage size this Stack Overflow answer is a good start.
We already know how much storage can __NSDictionaryI
have, but how does it know which size index to pick on initialization? The answer lies within the previously mentioned + __new:::::
class method. Converting some parts of the assembly back to C renders the following code:
1 2 3 4 5 6 7 8 9 10 |
|
The method looks linearly through __NSDictionaryCapacities
array until count
fits into the size. A quick glance in Hopper shows the contents of the array:
|
Converting to base-10 provides 0, 3, 6, 11, 19, 32, 52, 85 and so on. Notice that those are smaller numbers than the primes listed before. If you were to fit 32 key-value pairs into __NSDictionaryI
it will allocate space for 41 pairs, conservatively saving quite a few empty slots. This helps reducing the number of hash collisions, keeping the fetching time as close to constant as possible. Apart from trivial case of 3 elements, __NSDictionaryI
will never have its storage full, on average filling at most 62% of its space.
As a trivia, the last nonempty value of __NSDictionaryCapacities
is 0x11089481C742 which is 18728548943682 in base-10. You’d have to try really hard not to fit in within the pairs count limit, at least on 64-bit architecture.
If you were to use __NSDictionarySizes
in your code by declaring it as an extern
array, you’d quickly realize it’s not that easy. The code wouldn’t compile due to a linker error – the __NSDictionarySizes
symbol is undefined. Inspecting the CoreFoundation library with nm utility:
|
…clearly shows the symbols are there (for ARMv7, ARMv7s and ARM64 respectively):
|
Unfortunately the nm manual clearly states:
If the symbol is local (non-external), the symbol’s type is instead represented by the corresponding lowercase letter.
The symbols for __NSDictionarySizes
are simply not exported – they’re intended for internal use of the library. I’ve done some research to figure out if it’s possible to link with non-exported symbols, but apparently it is not (please tell me if it is!). We can’t access them. That is to say, we can’t access them easily.
Here’s an interesting observation: in both iOS 7.0 an 7.1 the kCFAbsoluteTimeIntervalSince1904
constant is laid out directly before __NSDictionarySizes
:
|
The best thing about kCFAbsoluteTimeIntervalSince1904
is that it is exported! We’re going to add 8 bytes (size of double
) to the address of this constant and reinterpret the result as pointer to NSUInteger
:
|
Then we can access its values by convenient indexing:
1 2 3 4 5 6 |
|
This hack is very fragile and will most likely break in the future, but this is the test project so it’s perfectly fine.
Now that we’ve discovered the internal structure of __NSDictionaryI
we can use this information to figure out why things work they way they work and what unforeseen consequences does the present implementation of __NSDictionaryI
introduce.
To make our investigation a little bit easier we will create a helper NSDictionary
category method that will print the contents of the instance
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 |
|
Let’s create a simple dictionary containing four values:
1 2 3 4 5 6 |
|
The output of the explored description is:
|
With that in mind let’s do a quick enumeration over dictionary:
1 2 3 |
|
And the output:
|
Enumeration seems to simply walk through the storage, ignoring the nil
keys and calling the block only for non-empty slots. This is also the case for fast enumeration, keyEnumerator
, allKeys
and allValues
methods. It makes perfect sense. The NSDictionary
is not ordered, so it doesn’t really matter in what sequence the keys and values are provided. Using the internal layout is the easiest and probably the fastest option.
__NSDictionaryI
may return something for nil
key!Let’s consider an example. Imagine we’re building a simple 3D strategy game set in space. The entire universe is split into cube-like sectors that imaginary factions can fight over. A sector can be referenced by its i
, j
and k
indexes. We shouldn’t use a 3D array to store the sectors info – the game space is huge and most of it is empty, so we would waste memory storing nil
pointers. Instead, we’re going to use a sparse storage in a form of NSDictionary
with a custom key class that will make it super easy to query if there is something at a given location.
Here’s the interface for key, a BC3DIndex
class:
1 2 3 4 5 6 7 |
|
And its equally trivial implementation:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 |
|
Notice how we’re being a good subclassing citizen: we implemented both isEqual:
and hash
methods and made sure that if two 3D-indexes are equal then their hash values are equal as well. The object equality requirements are fulfilled.
Here’s a trivia: what will the following code print?
1 2 3 4 5 |
|
It should be (null)
right? Nope:
|
To investigate this further let’s grab a dictionary’s description:
|
It turns out __NSDictionaryI
doesn’t check if the key
passed into objectForKey:
is nil
(and I’d argue this is a good design decision). Calling hash
method on nil
returns 0
, which causes the class to compare key at index 0
with nil
. This is important: it is the stored key that executes the isEqual:
method, not the passed in key.
The first comparison fails, since i
index for “A black hole!” is 2
whereas for nil
it’s zero. The keys are not equal which causes the dictionary to keep looking, hitting another stored key: the one for “Asteroids!”. This key has all three i
, j
, and k
properties equal to 0
which is also what nil
will return when asked for its properties (by the means of nil
check inside objc_msgSend
).
This is the crux of the problem. The isEqual:
implementation of BC3DIndex
may, under some conditions, return YES
for nil
comparison. As you can see, this is a very dangerous behavior that can mess things up easily. Always ensure that your object is not equal to nil
.
For the next two tests we’re going to craft a special key class that will have a configurable hash value and will print stuff to the console when executing hash
and isEqual:
method.
Here’s the interface:
1 2 3 4 5 6 7 |
|
And the implementation:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 |
|
This key is awful: we’re only equal to self, but we’re returning an arbitrary hash. Notice that this doesn’t break the equality contract.
isEqual
doesn’t have to be called to match the keyLet’s create a key and a dictionary:
1 2 |
|
The following call:
|
Prints this to the console:
|
As you can see, the isEqual:
method has not been called. This is very cool! Since the vast majority of keys out there are NSString
literals, they share the same address in the entire application. Even if the key is a very long literal string, the __NSDictionaryI
won’t run the potentially time consuming isEqual:
method unless it absolutely has to. And since 64-bit architectures introduced tagged pointers, some instances of NSNumber
, NSDate
and, apparently, NSIndexPath
benefit from this optimization as well.
Let’s create a very simple test case:
1 2 3 4 5 6 7 8 9 |
|
A single killer line:
|
Reveals the disaster:
|
This is extremely pathological case – every single key in the dictionary has ben equality tested. Even though each hash was different, it still collided with every other key, because the keys’ hashes were congruent modulo7, which turned out to be the storage size of the dictionary.
As mentioned before, notice that the last isEqual:
test is missing. The __NSDictionaryI
simply compared the pointers and figured out it must be the same key.
Should you care for this linear time fetching? Absolutely not. I’m not that into probabilistic analysis of hash distribution, but you’d have to be extremely lucky for all your hashes to be modulo congruent to dictionary’s size. Some collisions here and there will always happen, this is the nature of hash tables, but you will probably never run into the linear time issue. That is, unless you mess up your hash
function.
NSDictionary(字典)是使用 哈希表来实现key和value之间的映射和存储的, hash函数设计的好坏影响着数据的查找访问效率。数据在hash表中分布的越均匀,其访问效率越高。而在Objective-C中,通常都是利用NSString 来作为键值,其内部使用的hash函数也是通过使用 NSString对象作为键值来保证数据的各个节点在hash表中均匀分布。
- (void)setObject:(id)anObject forKey:(id <NSCopying>)aKey;�
NSDictionary中的key是唯一的,key可以是遵循NSCopying
协议和重载- (NSUInteger)hash;
、- (BOOL)isEqual:(id)object;
方法的任何对象。也就是说在NSDictionary内部,会对 aKey 对象 copy 一份新的。而 anObject 对象在其内部是作为强引用(retain或strong)。
hash
方法是用来计算该对象的 hash 值,最终的 hash 值决定了该对象在 hash 表中存储的位置。所以同样,如果想重写该方法,我们尽量设计一个能让数据分布均匀的 hash 函数。isEqual :
方法是为了通过 hash 值来找到 对象 在hash �表中的位置。在调用setObject: forKey:
后,内部会去调用 � key 对象的 hash
方法确定 object 在hash表内的入口位置,然后会调用 isEqual :
来确定该值是否已经存在于 NSDictionary中。
注:关于hash
,isEqual :
可以看看这篇文章 iOS开发 之 不要告诉我你真的懂isEqual与hash!
其实,NSDictionary使用NSMapTable实现。NSMapTable同样是一个key-value的容器,下面是NSMapTable的部分代码:
@interface NSMapTable : NSObject {
NSMapTableKeyCallBacks *keyCallBacks;
NSMapTableValueCallBacks *valueCallBacks;
NSUInteger count;
NSUInteger nBuckets;
struct _NSMapNode **buckets;
}
可以看出来NSMapTable是一个哈希+链表的数据结构,因此在NSMapTable中插入或者删除一对对象时
寻找的时间是O(1)+O(m),m最坏时可能为n。
O(1):为对key进行hash得到bucket的位置
O(m):遍历该bucket后面冲突的value,通过链表连接起来。
NSDictionary中的key Value遍历时是无序的,至如按照什么样的顺序,跟hash函数相关。NSMapTable使用NSObject的哈希函数。
-(NSUInteger)hash {
return (NSUInteger)self>>4;
}
上述是NSObject的哈希值的计算方式,简单通过移位实现。右移4位,左边补0.
因为对象大多存于堆中,地址相差4位应该很正常。
补充一个小知识:iOS setValue和setObject的区别:
- (void)setObject:(ObjectType)anObject forKey:(KeyType <NSCopying>)aKey;
- (void)setValue:(nullable ObjectType)value forKey:(NSString *)key;
首先:setObject: ForKey:
是NSMutableDictionary特有的;setValue: ForKey:
是KVC的主要方法。
区别:
(1)
setValue: ForKey:
的value是可以为nil的(但是当value为nil的时候,会自动调用removeObject:forKey方法);setObject: ForKey:
的value则不可以为nil。
(2)setValue: ForKey:
的key必须是不为nil的字符串类型;setObject: ForKey:
的key可以是不为nil的所有继承NSCopying的类型。