Python in order to enhance performance, actually use the shared economic

You may know, Python in order to improve the utilization efficiency of memory, using a memory allocation strategy a common set of objects.

For example, for those small digital object values ​​([-5, 256]), Boolean objects, None objects, the shorter string object (typically 20) and the like, an amount equal to the literal fact the same object object.

# 共用内存地址的例子
a = 100
b = 100
s = "python_cat"
t = "python_cat"

id(a) == id(b) # 结果:True
id(s) == id(t) # 结果:True

I once wrote an early age what “Python in the” privileged race “Yes? “These objects are collectively referred to as the” privileged race “, they are the optimization techniques used on the Python memory management mechanism.

Not long ago, I wrote a “little secret when the Python memory allocation”, also describes techniques for memory management.

These two articles differ: Old Man mainly related to the mechanism of memory shared with the object resides, and the new paper describes a memory allocation mechanisms and dynamic expansion memory recovery.

They made me automatically think of two words: shared economic and supply-demand balance.

If you have not read those two articles, I strongly suggest you look back, and then see if my association makes sense: that a few privileged class race is in fact shared memory objects, different objects on the surface, in fact, in recycling; As for Ye Hao understand the balance between supply and demand, create some objects that are expected to appeal to allocate memory, flexible adjustment at the time of expansion, to achieve a balance between supply and demand.

Look beyond, Python can be fun.

However, the interesting thing about Python is more than that, we want to continue to share another memory management mechanism, to some extent, it implements the integration and shared economic balance of supply and demand, from which we can reveal another identity of Python ……

1, immutable shared economic objects

“Privileged race” listed above are immutable objects (and the “supply and demand balance,” mainly in the variable objects), for these same objects, when there is multiple use, sharing an object it appears to be a pretty good optimization method.

I had the conjecture: Python are immutable objects may be privileged race.

I did not try to fully confirm it, we just want to visit one of the immutable object: the tuple. It is an immutable object, then, is there a mechanism to share with it?

Here it make some comparison with the list:

# 空对象的差别
a = []
b = []
c = ()
d = ()

print(id(a)==id(b))  # 结果:False
print(id(c)==id(d))  # 结果:True

Thus, two empty lists are different objects, and two empty tuple is actually the same object. This at least shows, only one empty tuple in memory, it is the prerogative of the race has been mentioned.

The experiment extends to the collection and dictionary, which is a variable object, you will find a list with the same result, there are multiple copies, that is not a privilege race. I am not an example of.

From the experimental results described above, but also leads to two questions, but they deviated from the subject of this article, I will not go into discrimination, simply out about:

    In addition to the empty tuple, what kind of tuples are “privileged race”? (PS: the number of elements, types, considering the size of the element itself, I would test a small area, we have not found so empty tuple is unique only.?)

    Compile-time and run-time distinction, which at the time before writing intern mechanism string ( “weakness Intern mechanism”) is also analyzed. (PS: print (id ([]) == id ([])), the result is True, the assignment of the first embodiment and then compare.)

2, shared economic variable objects

Empty tuple reflects a shared economy, but because it is immutable, so there is no dynamic expansion, it only reflects the balance between supply and demand very little.

As a control, a list of variable objects such as full performance of the supply and demand balance, it seems no way reflects the shared economy.

For example, we put a list can be thought of as a self-energizing cup (after all, it is some kind of container), then imagine its elements into different kinds of liquids (water, cola, wine ……).

So, our question is: if something can share a glass or two objects? Or, is it possible to share bird cup it? This saves memory (In that speak little secret article show off: “empty glass” can take up a lot of memory) to improve the efficiency of it.

For the first question, the answer is no, the verification process slightly. For the second question, in the previous section, we have verified two empty glasses (ie empty list), the answer is no.

However, the second question there are other possible! Let’s put it another experimental method:

# 实验版本:Python 3.6.1
a = [[] for i in range(4)]

for i in range(len(a)):
    print(f'{i} -- {id(a[i])}')
    # a[i] = 1 # PS:可去除注释,再执行一次,结果的顺序有差别

del a
print("after del")

b = [[] for i in range(4)]

for i in range(len(b)):
    print(f'{i} -- {id(b[i])}')

The above code in different environments, the results may vary. I once performed the following results:

0 -- 2012909395272
1 -- 2012909406472
2 -- 2012909395208
3 -- 2012909395144
after del
0 -- 2012909395272
1 -- 2012909406472
2 -- 2012909395208
3 -- 2012909395144

The results can be seen: the list of objects after being recovered, and will not completely eliminate, its memory address will be passed to the newly created list, that is, in fact, the list of the newly created shared memory address of the old list!

Combined with the previous example, we can say that two different lists will allocate memory has created statically, but, after a dynamic recovery, has created a list may be the same memory address! (Note: This is to say “probably” because before the new list creation, if there are other places to create a list, and that the latter may take away the initiative.)

Substantially extends to other variable objects, such set of dictionaries, have the same sharing policy, it is an object obvious: recycling “residual footer” These objects, memory fragmentation can be avoided to improve the execution performance.

Sharing a cup, a cup is better than to re-create, to be more efficient and convenient, right?

Python interpreter when implementing this mechanism, the use of global variables called free_list, and its working principle is:

    When you create a new object, check for available objects in free_list, there is removed using, do not create

    When these objects are analyzed configuration, it is checked whether there is space left free_list, wherein there is stored

    Certain types of objects into free_list, leaving only the “shell”, and emptied all of its internal elements (ie only share the cup, not shared alcoholism)

Source: https: //

Well, now we can say, lists, sets and dictionaries of these variable objects, they are not said before, racial privilege, but behind them are hidden share ideas recycled, which is interlinked.

Python interpreter in memory management took great pains ah, those on the base object commonplace, it exerts a lot of small magic, when we have no awareness of their orderly functioning, and when we finally seen clearly, it is not it’s not lament the exquisite.

Is a cheapskate, “The Economist” on the Python considered.

Recalling the full text, as a final summary:

    Smaller number, the shorter the string, Boolean value with an empty tuples and other immutable objects, they exist mechanism “sharing economy”, to enhance efficiency in the use of memory

    Lists, sets and dictionaries and other variable objects, which exist mechanism “supply and demand balance,” the pre-allocation and over-allocation, enhance the efficiency of memory allocation

    List and other objects, there are still shared “container housing” mechanism, recycling of idle resources, a comprehensive program to enhance performance

PS: more than half the time of this writing, I think it should be written “meow star guest” series, but after thinking it over, finally gave up the (mostly lazy). Their speculative forces and insight is the same strain, if you like this article, I recommend reading “meow star guest” series (two of which):

Myth Python object identity: are numbers to all citizens from all

Space boundary Python object: immune tolerance and openness

Public number [cat] Python, the serial number of high-quality series of articles, there are philosophical cat meow Star series, Python Advanced Series, recommended books series, technical writing, and recommend high-quality English translation and so on, welcome attention Oh.

Leave a Reply