Asked  7 Months ago    Answers:  5   Viewed   18 times

I have a problem with iterating on a file. Here's what I type on the interpreter and the result:

>>> f = open('baby1990.html', 'rU')
>>> for line in f.readlines():
...  print(line)
... 
# ... all the lines from the file appear here ...

When I try to iterate on the same open file again I get nothing!

>>> for line in f.readlines():
...  print(line)
... 
>>>

There is no output at all. To solve this I have to close() the file then open it again for reading! Is that normal behavior?

 Answers

89

Yes, that is normal behavior. You basically read to the end of the file the first time (you can sort of picture it as reading a tape), so you can't read any more from it unless you reset it, by either using f.seek(0) to reposition to the start of the file, or to close it and then open it again which will start from the beginning of the file.

If you prefer you can use the with syntax instead which will automatically close the file for you.

e.g.,

with open('baby1990.html', 'rU') as f:
  for line in f:
     print line

once this block is finished executing, the file is automatically closed for you, so you could execute this block repeatedly without explicitly closing the file yourself and read the file this way over again.

Tuesday, June 1, 2021
 
Besnik
answered 7 Months ago
83

Path.GetFileName

Returns the file name and extension of a file path that is represented by a read-only character span.


Path.GetFileNameWithoutExtension

Returns the file name without the extension of a file path that is represented by a read-only character span.


The Path class is wonderful.

Tuesday, June 1, 2021
 
Exoon
answered 7 Months ago
31

The problem (as seen here http://forums.asp.net/t/1060363.aspx) seems to be when you use the visibility property on the surrounding panel (as it seems you are from the linked question).

The suggested workaround is to use CSS visibility instead so use this to make it invisible -

<asp:Panel ID="pnlUpload" runat="server" class="workerDetailsPanelLeft" style="display:none">

The explanation for this from the thread is

If your container is set to invisible, the upload control is not actually rendered as HTML, causing the form's enctype not to be set to enctype="multipart/form-data", causing the file upload control not to post the selected file back to the server. The workaround is either to make sure the FileUpload control is rendered to HTML (by setting its style to display:none in stead of Visible=false), or by manually setting the enctype

So another workaround would be to alter your form tag to this

<form id="form1" enctype="multipart/form-data" runat="server">

I think either one of those should solve your problem.

Tuesday, August 10, 2021
 
user1865027
answered 4 Months ago
43

You may keep local file file_pc as is (pipes.quote will escape the spaces). The remote file should be changed:

import pipes

file_pi = 'pi@192.168.X.X:/home/pi/folder/file with space.smth'
host, colon, path = file_pi.partition(':')
assert colon
file_pi = host + colon + pipes.quote(path)

i.e., user@host:/path/with space should be changed to user@host:'/path/with space'

Thursday, September 2, 2021
 
RenegadeAndy
answered 3 Months ago
44

The problem seems to be that multiprocessing has a limit to the largest int it can pass to subprocesses inside an xrange. Here's a quick test:

import sys
from multiprocessing import Pool
def doit(n):
  print n
if __name__ == "__main__":
  procs = int(sys.argv[1])
  iters = int(float(sys.argv[2]))
  p = Pool(processes=procs)
  for points in p.map(doit, [xrange(int(iters))] * procs):
    pass

Now:

$ ./multitest.py 2 1E8
xrange(100000000)
xrange(100000000)
$ ./multitest.py 2 1E9
xrange(1000000000)
xrange(1000000000)
$ ./multitest.py 2 1E10
xrange(1410065408)
xrange(1410065408)

This is part of a more general problem with multiprocessing: It relies on standard Python pickling, with some minor (and not well documented) extensions to pass values. Whenever things go wrong, the first thing to check is that the values are arriving the way you expected.

In fact, you can see this problem by playing with pickle, without even touching multiprocessing (which isn't always the case, because of those minor extensions, but often is):

>>> pickle.dumps(xrange(int(1E9)))
'c__builtin__nxrangenp0n(I0nI1000000000nI1ntp1nRp2n.'
>>> pickle.dumps(xrange(int(1E10)))
'c__builtin__nxrangenp0n(I0nI1410065408nI1ntp1nRp2n.'

Even without learning all the details of the pickle protocol, it should be obvious that the I1000000000 in the first case is 1E9 as an int, while the equivalent chunk of the next case is about 1.41E9, not 1E10, as an int. You can experiment

One obvious solution to try is to pass int(iters) instead of xrange(int(iters)), and let calculate_pi create the xrange from its argument. (Note: In some cases an obvious transformation like this can hurt performance, maybe badly. But in this case, it's probably slightly better if anything—a simpler object to pass, and you're parallelizing the xrange construction—and of course the difference is so tiny it probably won't matter. Just make sure to think before blindly transforming.)

And a quick test shows that this now works:

import sys
from multiprocessing import Pool

def doit(n):
  print xrange(n)

if __name__ == "__main__":
    procs = int(sys.argv[1])
    iters = int(float(sys.argv[2]))
    p = Pool(processes=procs)
    for points in p.map(doit, [iters] * procs):
      pass

Then:

$ ./multitest.py 2 1E10
xrange(10000000000)
xrange(10000000000)

However, you will still run into a larger limit:

$ ./multitest.py 2 1E100
OverflowError: Python int too large to convert to C long

Again, it's the same basic problem. One way to solve that is to pass the arg all the way down as a string, and do the int(float(a)) inside the subprocesses.

As a side note: The reason I'm doing iters = int(float(sys.argv[2])) instead of just iters = float(sys.argv[2]) and then using int(iters) later is to avoid accidentally using the float iters value later on (as the OP's version does, in computing total and therefore total_in / total).

And keep in mind that if you get to big enough numbers, you run into the limits of the C double type: 1E23 is typically 99999999999999991611392, not 100000000000000000000000.

Friday, October 8, 2021
 
Don
answered 2 Months ago
Don
Only authorized users can answer the question. Please sign in first, or register a free account.
Not the answer you're looking for? Browse other questions tagged :  
Share