Asked  7 Months ago    Answers:  5   Viewed   34 times

Does it matter how many files I keep in a single directory? If so, how many files in a directory is too many, and what are the impacts of having too many files? (This is on a Linux server.)

Background: I have a photo album website, and every image uploaded is renamed to an 8-hex-digit id (say, a58f375c.jpg). This is to avoid filename conflicts (if lots of "IMG0001.JPG" files are uploaded, for example). The original filename and any useful metadata is stored in a database. Right now, I have somewhere around 1500 files in the images directory. This makes listing the files in the directory (through FTP or SSH client) take a few seconds. But I can't see that it has any effect other than that. In particular, there doesn't seem to be any impact on how quickly an image file is served to the user.

I've thought about reducing the number of images by making 16 subdirectories: 0-9 and a-f. Then I'd move the images into the subdirectories based on what the first hex digit of the filename was. But I'm not sure that there's any reason to do so except for the occasional listing of the directory through FTP/SSH.

 Answers

27

FAT32:

  • Maximum number of files: 268,173,300
  • Maximum number of files per directory: 216 - 1 (65,535)
  • Maximum file size: 2 GiB - 1 without LFS, 4 GiB - 1 with

NTFS:

  • Maximum number of files: 232 - 1 (4,294,967,295)
  • Maximum file size
    • Implementation: 244 - 26 bytes (16 TiB - 64 KiB)
    • Theoretical: 264 - 26 bytes (16 EiB - 64 KiB)
  • Maximum volume size
    • Implementation: 232 - 1 clusters (256 TiB - 64 KiB)
    • Theoretical: 264 - 1 clusters (1 YiB - 64 KiB)

ext2:

  • Maximum number of files: 1018
  • Maximum number of files per directory: ~1.3 × 1020 (performance issues past 10,000)
  • Maximum file size
    • 16 GiB (block size of 1 KiB)
    • 256 GiB (block size of 2 KiB)
    • 2 TiB (block size of 4 KiB)
    • 2 TiB (block size of 8 KiB)
  • Maximum volume size
    • 4 TiB (block size of 1 KiB)
    • 8 TiB (block size of 2 KiB)
    • 16 TiB (block size of 4 KiB)
    • 32 TiB (block size of 8 KiB)

ext3:

  • Maximum number of files: min(volumeSize / 213, numberOfBlocks)
  • Maximum file size: same as ext2
  • Maximum volume size: same as ext2

ext4:

  • Maximum number of files: 232 - 1 (4,294,967,295)
  • Maximum number of files per directory: unlimited
  • Maximum file size: 244 - 1 bytes (16 TiB - 1)
  • Maximum volume size: 248 - 1 bytes (256 TiB - 1)
Tuesday, June 1, 2021
 
SuperString
answered 7 Months ago
35

Well, I guess now that the oracle tag was added, I'll add the oracle answer.

$rows = array(row, row, row, ...); // from oracle sql results
$first_24_rows = array_slice($rows, 0, 24);
Saturday, May 29, 2021
 
KouiK
answered 7 Months ago
26

Hadley explains this on pp. 99; 133 of his ggplot2 book (1st edition), or pp. 160 - 161 if you have the second edition.

The issue is that, as you say, limits inside the scale or setting ylim() causes data to be thrown away, as they are constraining the data. For a true zoom (keep all the data), you need to set the limits inside of the Cartesian coordinate system (or other coordinate systems https://ggplot2.tidyverse.org/reference/#section-coordinate-systems). For more see: http://docs.ggplot2.org/current/coord_cartesian.html

ggplot(d, aes(x, y, group=grp)) + 
    geom_line() + 
    coord_cartesian(ylim=c(0, 7))

enter image description here

Monday, June 7, 2021
 
Nate
answered 7 Months ago
39

In response to your comment "so how many times do i know to run it?", this example runs until it lists all the files whose names match strPattern. Change the strFolder constant.

Public Sub ListESY()
Const strFolder As String = "C:SomeFolder"
Const strPattern As String = "*.ESY"
Dim strFile As String
strFile = Dir(strFolder & strPattern, vbNormal)
Do While Len(strFile) > 0
    Debug.Print strFile '<- view this in Immediate window; Ctrl+g will take you there
    strFile = Dir
Loop
End Sub
Friday, July 30, 2021
 
letrollpoilu
answered 5 Months ago
83

It might, but any time the corresponding files in export and webroot have the same content but different modification times, you'd wind up performing an unnecessary copy operation. You'd probably get slightly smarter behavior from rsync:

rsync -pr ./export /path/to/webroot

Besides, rsync can copy files from one host to another over an SSH connection, if you ever have a need to do that. Plus, it has a zillion options you can specify to tweak its behavior - look in the man page for details.

EDIT: with respect to your clarification about what you mean by preserving permissions: you'd probably want to leave off the -p option.

Thursday, September 9, 2021
 
yinka
answered 3 Months ago
Only authorized users can answer the question. Please sign in first, or register a free account.
Not the answer you're looking for? Browse other questions tagged :  
Share