Full Unicode support
This is the biggest change. In order to achieve this I had to refactor all functions and internals either returning or accepting a string. Incidentally this helped me having a better understanding of how Unicode works and how it should be handled at the C level in terms of differences between Python 2 and 3. Issue #1040 includes all the reasonings I've been through and potentially serves as a documentation for people who are facing a similar task (handling Unicode in C for both Python 2 and 3). Up until version 5.2.x psutil functions returning a string had different problems as they could:
- raise decoding error on Python 3 in case of non-ASCII string
- return unicode instead of str (Python 2)
- return incorrect / invalid encoded data in case of non-ASCII string
- all strings are encoded by using the OS filesystem encoding (sys.getfilesystemencoding()) which varies depending on the platform (e.g. "UTF-8" on OSX, "mbcs" on Win)
- no API call is supposed to crash with UnicodeDecodeError
- instead, in case of badly encoded data returned by the OS, the following error handlers are used to replace the corrupted characters in the string:
- Python 3: sys.getfilesystemencodeerrors() (PY 3.6+) or "surrogatescape"`` on POSIX and "replace" on Windows
- Python 2: "replace"
- on Python 2 all APIs return bytes (str type), never unicode
- on Python 2 you can go back to unicode by doing:
>>> unicode(proc.exe(), sys.getdefaultencoding(), errors="replace")
Improved process_iter() function
process_iter() accepts two new parameters in order to invoke Process.as_dict() internally: "attrs" and "ad_value". With this you can iterate over all processes in one shot without having to catch NoSuchProcess explicitly. Before:>>> import psutil
>>> for proc in psutil.process_iter():
... try:
... pinfo = proc.as_dict(attrs=['pid', 'name'])
... except psutil.NoSuchProcess:
... pass
... else:
... print(pinfo)
...
{'pid': 1, 'name': 'systemd'}
{'pid': 2, 'name': 'kthreadd'}
{'pid': 3, 'name': 'ksoftirqd/0'}
...
Now:>>> import psutil
>>> for proc in psutil.process_iter(attrs=['pid', 'name']):
... print(proc.info)
...
{'pid': 1, 'name': 'systemd'}
{'pid': 2, 'name': 'kthreadd'}
{'pid': 3, 'name': 'ksoftirqd/0'}
This improves expressiveness as it makes it possible to use nice list/dict comprehensions. Here's some examples.Processes having "python" in their name::
>>> from pprint import pprint as pp
>>> pp([p.info for p in psutil.process_iter(attrs=['pid', 'name']) if 'python' in p.info['name']])
[{'name': 'python3', 'pid': 21947},
{'name': 'python', 'pid': 23835}]
Processes owned by user::
>>> import getpass >>> pp([(p.pid, p.info['name']) for p in psutil.process_iter(attrs=['name', 'username']) if p.info['username'] == getpass.getuser()]) (16832, 'bash'), (19772, 'ssh'), (20492, 'python')]Processes actively running::
>>> pp([(p.pid, p.info) for p in psutil.process_iter(attrs=['name', 'status']) if p.info['status'] == psutil.STATUS_RUNNING])
[(1150, {'name': 'Xorg', 'status': 'running'}),
(1776, {'name': 'unity-panel-service', 'status': 'running'}),
(20492, {'name': 'python', 'status': 'running'})]
Automatic overflow handling of numbers
On very busy or long-lived system systems numbers returned by disk_io_counters() and net_io_counters() functions may wrap (restart from zero). Up to version 5.2.x you had to take this into account while now this is automatically handled by psutil (see: #802). If a "counter" restarts from 0 psutil will add the value from the previous call for you so that numbers will never decrease. This is crucial for applications monitoring disk or network I/O in real time. Old behavior can be resumed by passing nowrap=True argument.SunOS Process environ()
Process.environ() is now available also on SunOS (see #1091).Other improvements and bug fixes
Amongst others, here's a couple of important bug fixes I'd like to mention:- #1044: on OSX different Process methods could incorrectly raise AccessDenied for zombie processes. This was due to poor proc_pidpath OSX API.
- #1094: on Windows, pid_exists() may lie due to the poor OpenProcess Windows API which can return a handle even when a process PID no longer exists. This had repercussions for many Process methods such as cmdline(), environ(), cwd(), connections() and others which could have unpredictable behaviors such as returning empty data or erroneously raise NoSuchProcess exceptions. For the same reason (broken OpenProcess API), processes could unexpectedly stick around after being terminate()d and wait()ed on.
