Asked  8 Months ago    Answers:  5   Viewed   34 times

I have a script which, each time is called, gets the first line of a file. Each line is known to be exactly of the same length (32 alphanumeric chars) and terminates with "rn". After getting the first line, the script removes it.

This is done in this way:

$contents = file_get_contents($file));
$first_line = substr($contents, 0, 32);
file_put_contents($file, substr($contents, 32 + 2)); //+2 because we remove also the rn

Obviously it works, but I was wondering whether there is a smarter (or more efficient) way to do this?

In my simple solution I basically read and rewrite the entire file just to take and remove the first line.

 Answers

76

There is no more efficient way to do this other than rewriting the file.

Wednesday, March 31, 2021
 
Sujith
answered 8 Months ago
56

It happens because PHP CS Fixer modifies the file, as @LazyOne said, but you have empty value of "Output paths to refresh" so IDE cannot know about these changes.

Set the value of "Output paths to refresh" to $FileName$ - same as in arguments - to make PhpStorm aware about the changes (it depends on the "Working directory" value which has been set in Other Options - if it is set to $FileDir$ then you don't need to mention it in the paths to refresh).

Wednesday, March 31, 2021
 
Terry
answered 8 Months ago
36

docs for io module

with open(fname, 'rb') as fh:
    first = next(fh).decode()

    fh.seek(-1024, 2)
    last = fh.readlines()[-1].decode()

The variable value here is 1024: it represents the average string length. I choose 1024 only for example. If you have an estimate of average line length you could just use that value times 2.

Since you have no idea whatsoever about the possible upper bound for the line length, the obvious solution would be to loop over the file:

for line in fh:
    pass
last = line

You don't need to bother with the binary flag you could just use open(fname).

ETA: Since you have many files to work on, you could create a sample of couple of dozens of files using random.sample and run this code on them to determine length of last line. With an a priori large value of the position shift (let say 1 MB). This will help you to estimate the value for the full run.

Wednesday, June 2, 2021
 
ManojGeek
answered 5 Months ago
37

I've started project called Big File Tools. It is proven to work on Linux, Mac and Windows (even 32-bit variants). It provides byte-precise results even for huge files (>4GB). Internally it uses brick/math - arbitrary-precision arithmetic library.

Install it using composer.

composer install jkuchar/BigFileTools

and use it:

<?php
$file = BigFileToolsBigFileTools::createDefault()->getFile(__FILE__);
echo $file->getSize() . " bytesn";

Result is BigInteger so you can compute with results

$sizeInBytes = $file->getSize();
$sizeInMegabytes = $sizeInBytes->toBigDecimal()->dividedBy(1024*1024, 2, BrickMathRoundingMode::HALF_DOWN);    
echo "Size is $sizeInMegabytes megabytesn";

Big File Tools internally uses drivers to reliably determine exact file size on all platforms. Here is list of available drivers (updated 2016-02-05)

| Driver           | Time (s) ?          | Runtime requirements | Platform 
| ---------------  | ------------------- | --------------       | ---------
| CurlDriver       | 0.00045299530029297 | CURL extension       | -
| NativeSeekDriver | 0.00052094459533691 | -                    | -
| ComDriver        | 0.0031449794769287  | COM+.NET extension   | Windows only
| ExecDriver       | 0.042937040328979   | exec() enabled       | Windows, Linux, OS X
| NativeRead       | 2.7670161724091     | -                    | -

You can use BigFileTools with any of these or fastest available is chosen by default (BigFileTools::createDefault())

 use BigFileToolsBigFileTools;
 use BigFileToolsDriver;
 $bigFileTools = new BigFileTools(new DriverCurlDriver());
Wednesday, June 2, 2021
 
qitch
answered 5 Months ago
33

Does this do it for you? Good old Descendants property.

string xmlInput = ...;
XDocument myDoc = XDocument.Parse(xmlInput);
//
List<XElement> someElements = myDoc.Descendants("a").ToList();
someElements.ForEach(x => x.Value = "Foo");
//
Console.WriteLine(myDoc);

Hmm, I see you have an attribute in there. Can do that too:

string xmlInput = //...
XDocument myDoc = XDocument.Parse(xmlInput);
//
List<XText> someText =
  myDoc.Descendants()
  .Nodes()
  .OfType<XText>()
  .Where(x => x.Value.StartsWith("{") && x.Value.EndsWith("}"))
  .ToList();
//
List<XAttribute> someAttributes =
  myDoc.Descendants()
  .Attributes()
  .Where(x => x.Value.StartsWith("{") && x.Value.EndsWith("}"))
  .ToList();
//
someText.ForEach(x => x.Value = "Foo");
someAttributes.ForEach(x => x.Value = "Bar");
//
Console.WriteLine(myDoc);

Ah, now with what you're expecting, I will make it work:

List<XElement> e = myDoc.Descendants("a").ToList();
e.Where(x => x.Attribute("name").Value == "username").Single().Value = "abc";
e.Where(x => x.Attribute("name").Value == "password").Single().Value = "abc";
Tuesday, August 31, 2021
 
Mark116
answered 2 Months ago
Only authorized users can answer the question. Please sign in first, or register a free account.
Not the answer you're looking for? Browse other questions tagged :
 
Share