On 17 March 2010 19:37, Tim Jenness <[log in to unmask]> wrote:
>
> On Mar 10, 2010, at 12:13 AM, David Berry wrote:
>
>> On 10 March 2010 08:58, Tim Jenness <[log in to unmask]> wrote:
>>> These errors occur during bolomap or itermap writing and so a huge amount of
>>> memory is in use.
>>>
>>> Maybe one should just take the code from tcsh. Glob system call does not
>>> translate environment variables.
>>
>> The wordexp system call seems to do most of what we would want. From
>> the man page...
>>
>> The expansion done consists of the following stages: tilde expansion
>> (replacing ~user by user's home directory), variable substitution
>> (replacing $FOO by the value of the environment variable FOO), command
>> substitution (replacing $(command) or 'command' by the output of com-
>> mand), arithmetic expansion, field splitting, wildcard expansion, quote
>> removal.
>>
>> The result of expansion of special parameters ($@, $*, $#, $?, $-, $$,
>> $!, $0) is unspecified.
>>
>> Field splitting is done using the environment variable $IFS. If it is
>> not set, the field separators are space, tab and newline.
>>
>
> That sounds great and I think that ONE_FIND_FILE could be very much simplified by rewriting it to use wordexp(). Unfortunately ONE_SHELL_ECHO is the routine that is causing the initial problem and it explicitly states that globbing is disabled and only shell metacharacters are expanded. It doesn't look like wordexp() can handle that.
Off the top of my head, I can't think why the NDG routines that use
ONE_SHELL_ECHO (i.e. the ones that create new output NDFs) should ever
end up in a position where the output NDF name contains wild cards.
And if there are no wild cards in the output NDF name, then it won't
make any difference whether globbing is performed or not.
David
> Also, HDS is forking (sometimes) in rec1_find_file (those shell scripts are pretty crazy). We could mitigate that somewhat by making it fork a shell process at startup and then reuse it subsequently. Would require a bit of work and NDG currently disables shell mode in HDS explicitly so that it can call ONE_SHELL_ECHO (something to do with NDG handling spaces in file names but HDS not doing so when shell expansion is used).
>
> Finally, I note that rec1_get_path only calls rec1_find_file if there are metacharacters to expand so the approach may be to:
>
> 1. Move the code from rec1_get_path that looks at metacharacters to starutil (obviously changing HDS to use it)
> 2. Modify one_shell_echo so that it only forks if there are metacharacters
> 3. Modify one_find_file to use wordexp (should be drop in replacement for forking)
> 4. Create a new HDS_SHELL that will use wordexp instead of a real shell.
>
> The MAKEMAP case usually does not involve metacharacters so this should solve the bulk of the problem.
>
> --
> Tim Jenness
> Joint Astronomy Centre
>
>
>
>
>
|