["python","file-io","filesystems","queue","cluster-computing"]
"OSError: [Errno 17] File exists" when trying to use os.makedirs

I have several threads running in parallel from Python on a cluster system. Each python thread outputs to a directory mydir. Each script, before outputting checks if mydir exists and if not creates it: if not os.path.isdir(mydir): os.makedirs(mydir) but this yields the error: os.makedirs(self.log_dir) File "/usr/lib/python2.6/os.py", line 157, in makedirs mkdir(name,mode) OSError: [Errno 17] File exists I suspect it might be due to a race condition, where one job creates the dir before the other gets to it. Is this possible? If so, how can this error be avoided? I'm not sure it's a race condition so was wondering if other issues in Python can cause this odd error.
Any time code can execute between when you check something and when you act on it, you will have a race condition. One way to avoid this (and the usual way in Python) is to just try and then handle the exception while True: mydir = next_dir_name() try: os.makedirs(mydir) break except OSError, e: if e.errno != errno.EEXIST: raise # time.sleep might help here pass If you have a lot of threads trying to make a predictable series of directories this will still raise a lot of exceptions, but you will get there in the end. Better to just have one thread creating the dirs in that case