hi Eygene,

thanks for all the input, but I don't think we're heading in the right 
direction here ...
I've compiled the file and ran it on one of our resource brokers:

./pathconf  /share/gridmapdir/dteam000 /tmp/rpms.boszwijn
'/share/gridmapdir/dteam000': _PC_NAME_MAX = 255.
'/tmp/rpms.boszwijn': _PC_NAME_MAX = 255.

(/share/gridmapdir is the offending directory where 'readdir' crashes) 
so PC_NAME_MAX seems to be 255 for local and NFS files (which , 
incidentally, is the default for stock RH kernels, IIRC).

Maarten, did you see my private email with the threading opendir/readdir 
sample?

regards,

Jan Just Keijser
NIKHEF
Amsterdam


Eygene Ryabinkin wrote:
> Maarten, good day.
>
> Wed, Apr 25, 2007 at 03:35:09PM +0200, [log in to unmask] wrote:
>   
>> I meant to say the max. length of the d_name field may be larger.
>>     
>
> Precisely. And there is no good way to determine it at the
> compilation time.
>
>   
>>> You need to allocate space for the struct with something like:
>>>   struct dirent *buffer;
>>>   size = dirent_buf_size(gridmapdirstream);
>>>   buffer = (struct dirent *)malloc(size);
>>> you aren't supposed to use the struct without allocating enough space
>>> for it.
>>>
>>> Have a look at:
>>> http://seclists.org/fulldisclosure/2005/Nov/0001.html
>>> for the dirent_buf_size function and the reason why you can not just
>>> use something like:
>>>   pathmax = pathconf("foo", _PC_NAME_MAX);
>>>   buffer = (struct dirent *)malloc(offsetof(struct dirent, d_name) + pathmax + 1);
>>>       
>> Since we have plenty of memory, what do you think of this:
>>
>>     char buf[sizeof(struct dirent) + PATH_MAX + 1];
>>     
>
> And if the maximal length of dirent->d_name at the NIKHEF machine
> (or some other host) will be longer than 2*PATH_MAX? This can help
> in the particular case, but it is not the general way to fix this
> issue if it lies in the size of the 'struct dirent'.
>
> Note, that the usage of the 'pathconf' function makes us safe for
> any filesystem that correctly reports its maximal number of bytes
> in the pathlen. And this is done in the run-time.  You can use
> something like 'sizeof(struct dirent) + pathconf(...) + 1', but use
> malloc, since we need the run-time memory allocation.
>
> Interesting issue: I will try to see how the readdir is implemented.
> It should have the same problem, but it should be handled internally.
>
> I assume that the root of the problem is that NFS names can be
> longer than the local ones? No? For the test I am attaching the
> very simple C program that prints the results of the pathconf() for
> various directories and files. I am getting the same results for
> the local files and NFS mounts on our cluster. Ronald, can you try
> the program with the names of your grid-mapfile and some local file?
>   
> ------------------------------------------------------------------------
>
> /*
>  * gcc -o pathconf pathconf.c
>  * Usage: pathconf path1 [path2] [path3] [...]
>  */
>
> #include <stdio.h>
> #include <unistd.h>
>
> int
> main(int argc, char *argv[])
> {
> 	if (argc < 2) {
> 		fprintf(stderr, "Should give at least one filename.\n");
> 		exit(1);
> 	}
> 	argv++;
> 	while (*argv) {
> 		printf("'%s': _PC_NAME_MAX = %ld.\n", *argv,
> 		    pathconf(*argv, _PC_NAME_MAX));
> 		argv++;
> 	}
> 	return 0;
> }
>