bash oom problem

[psmith]

On 04/10/09 14:03, Patrick O'Callaghan wrote:
> On Sun, 2009-10-04 at 13:54 +0100, psmith wrote:
>    
>> On 04/10/09 11:17, Sharpe, Sam J wrote:
>>      
>>> 2009/10/4 psmith<psmith at fedoraproject.org>:
>>>
>>>        
>>>> hi list, whilst trying to run this bash command
>>>>
>>>> for w in {A..Z}{A..Z}{A..Z}{A..Z}{A..Z}{A..Z}{A..Z}{A..Z} ;do echo $w;done>
>>>> wl1
>>>>
>>>>          
>>> Consider that for a second...
>>>
>>> You are trying to generate a list of all possible combinations of an 8
>>> character word composed of only uppercase letters - that's 26^8
>>> combinations (208 Billion). Each word is 8 bytes long, which I make to
>>> be 1670616516608 bytes... or to put it another way, 1.5 TB
>>>
>>> So, you've got 4 GB of virtual memory and you are trying to fit an
>>> 1555GB array into it. Simple mathematics says no. Dumping the
>>> arguments before it dies is pointless, because it hasn't even got as
>>> far as expanding arguments yet.
>>>
>>> You need to think of another was to do this and I humbly suggest that
>>> Bash should not be high on your list.
>>>
>>>
>>>        
>> thanks for the reality check sam :o
>>
>> do you have any suggestions what/where i should be looking to get my
>> required output?
>>      
> As Sam says, your initial attempt is trying to produce all 1.5TB at
> once, which is why it blows up, but that's going to happen anyway even
> if you program the same thing in C. IOW it's not an issue with the Shell
> per se, but with the approach to the problem.
>
> Assuming you actually want all 1.5TB of data (really?) you'd be better
> consuming it as it's produced, perhaps via a pipe. Of course that
> depends on what you're doing with it, which you haven't said.
>
> poc
>
>    
i'm doing some pen testing of my brother's companies network he wants me 
to see if it's possible to get in so I'd be using the output as a word 
list, and yes unfortunately i'll need all of it for comparison i'm not 
sure if the program (aircrack) takes data from a pipe as a word list, 
but i'll look into it

phil

phil