Does Mysql / Python Benefit From The Cell Processor?

Discuss software development issues here.

Re: Does Mysql / Python Benefit From The Cell Processor?

Postby ppietro » 27 Jan 2009, 06:36

jintal wrote:How about this? http://www.linuxjournal.com/article/8497
I'm thinking... write short PPE and SPE C code, wherein the SPE one has the Python interpreter embedded in it. Any good?


No - that probably won't work.

I had to read thru that article carefully. The way I understand it, what the author is doing is linking the standard Python interpreter into the C program. So - the python interpreter would be the stock PPE interpreter, not a customized SPE interpreter.

So - unless you re-compile the Python interpreter to run on the SPE - this won't work. And - based on the SPEs instruction set, it's highly unlikely it will compile.

That's the next question you should look at - how complete is SPE C/C++? My guess is that it's a subset of standard C/C++.

And - even if it does work - the python interpreter might be too large to run in an SPE memory space. According to Wikipedia: "With the current generation of the Cell, each SPE contains a 256 KiB embedded SRAM for instruction and data"

So - 256 Kb per SPE - the code has to be pretty tight to fit in there. :D

You might want to check out CorePy, like you suggested earlier:
http://www.corepy.org/wiki/index.php?ti ... y_Cell/SPU

Cheers,
Paul
User avatar
ppietro
Site Admin
Site Admin
 
Posts: 4965
Joined: 13 Sep 2007, 22:18

Re: Does Mysql / Python Benefit From The Cell Processor?

Postby billb » 27 Jan 2009, 15:18

Hey Paul,

What do you make of this?

http://forums.ps2dev.org/viewtopic.php?t=11188

Looks like a Java VM for SPE?
PS3 60GB [CECHA01], FW 3.15, YDL 6.2, Samsung T260HD @ 1920x1200
Powermac G4 1.25 GHz x2, 2 GB RAM, YDL 6.2
User avatar
billb
Site Admin
Site Admin
 
Posts: 5522
Joined: 24 May 2007, 20:30
Location: Eastern NC, USA

Re: Does Mysql / Python Benefit From The Cell Processor?

Postby jintal » 27 Jan 2009, 21:26

Question... so basically, any C code that can be successfully compiled with the spe compiler (spu-gcc) will be used by the spe? :?
jintal
ydl newbie
ydl newbie
 
Posts: 11
Joined: 23 Jan 2009, 12:55

Re: Does Mysql / Python Benefit From The Cell Processor?

Postby ppietro » 27 Jan 2009, 22:43

jintal wrote:Question... so basically, any C code that can be successfully compiled with the spe compiler (spu-gcc) will be used by the spe? :?


Well - kinda.

What you have to do is compile a SPE "package" (that's my term - not official) and use the PPE to "load" it into an SPE's 256 kB memory, then tell the SPE to execute it. The SPE will then finish its calculations and come up with some output data. Depending on the SPE code, the SPE will either write that data back to main memory via DMA or the PPE can query the SPE for the finished data.

Cheers,
Paul
User avatar
ppietro
Site Admin
Site Admin
 
Posts: 4965
Joined: 13 Sep 2007, 22:18

Re: Does Mysql / Python Benefit From The Cell Processor?

Postby ppietro » 28 Jan 2009, 01:20

billb wrote:Hey Paul,

What do you make of this?

http://forums.ps2dev.org/viewtopic.php?t=11188

Looks like a Java VM for SPE?


Yes it is - and it's genius. What it does is put a small, lightweight JVM (Java Virtual Machine) on each of the SPEs, then uses them to execute Java threads. Any Java opcode the SPE can't handle - since they're vector math processors, not scalar - are handled by the PPE, which kind of acts like a shell around the whole thing.

Of course, this means you have to use Java threads - which isn't exactly trivial. But - I betcha the Swing & AWT graphics libraries would take advantage of this.

If only I could find a download. They link to one in the post, but I'd rather get it from the author directly.

See his paper here:
http://base.google.com/base/a/2277898/D ... 5174124134

Cheers,
Paul
User avatar
ppietro
Site Admin
Site Admin
 
Posts: 4965
Joined: 13 Sep 2007, 22:18

Re: Does Mysql / Python Benefit From The Cell Processor?

Postby ppietro » 28 Jan 2009, 07:45

Hi everyone,

By the way - looking around the net, I found a reference to this program: spu-top

This program lets you see if the SPUs are being used or not. It's kind of like a Cell specific version of top

It should already be installed on your YDL 6.1 system. Just open up a terminal window and type: spu-top

Cheers,
Paul
User avatar
ppietro
Site Admin
Site Admin
 
Posts: 4965
Joined: 13 Sep 2007, 22:18

Re: Does Mysql / Python Benefit From The Cell Processor?

Postby billb » 28 Jan 2009, 17:30

ppietro wrote:This program lets you see if the SPUs are being used or not. It's kind of like a Cell specific version of top


Neat ... I tried it with the SPE scaler tool running, scaling mame 2x, and got this:

Spu(s): 0.0%us, 16.7%sys, 0.0%wait, 83.3%idle

:mrgreen:

Also, I read that paper on the Java CellVM and now my brain hurts. :P
PS3 60GB [CECHA01], FW 3.15, YDL 6.2, Samsung T260HD @ 1920x1200
Powermac G4 1.25 GHz x2, 2 GB RAM, YDL 6.2
User avatar
billb
Site Admin
Site Admin
 
Posts: 5522
Joined: 24 May 2007, 20:30
Location: Eastern NC, USA

Re: Does Mysql / Python Benefit From The Cell Processor?

Postby ppietro » 28 Jan 2009, 18:48

billb wrote:Neat ... I tried it with the SPE scaler tool running, scaling mame 2x, and got this:

Spu(s): 0.0%us, 16.7%sys, 0.0%wait, 83.3%idle

:mrgreen:

Also, I read that paper on the Java CellVM and now my brain hurts. :P


If you have PS3GRID installed, it's fun to watch it load up all 6:

Code: Select all
spu-top: Context View
Cpu(s) load avg: 2.73, 0.99, 0.46
Spu(s) load avg: 3.79, 1.09, 0.37
Cpu(s): 55.5%us, 28.5%sys,  0.0%wait,  0.0%nice, 16.1%idle
Spu(s):  0.0%us,100.0%sys,  0.0%wait,  0.0%idle

   PID   TID USERNAME   S F  %SPU SPE     TIME BINARY
  4679  4690 paulp      S     4.7   0   59.427 cellmd2_5.03_po
  4679  4689 paulp      S     4.7   1   59.465 cellmd2_5.03_po
  4679  4688 paulp      S     4.6   2   59.474 cellmd2_5.03_po
  4679  4687 paulp      S     4.7   3   59.481 cellmd2_5.03_po
  4679  4686 paulp      S     4.7   4   59.486 cellmd2_5.03_po
  4679  4685 paulp      S     4.7   5   59.492 cellmd2_5.03_po


Yeah - that Java paper is something, isn't it? :D

Cheers,
Paul
User avatar
ppietro
Site Admin
Site Admin
 
Posts: 4965
Joined: 13 Sep 2007, 22:18

Re: Does Mysql / Python Benefit From The Cell Processor?

Postby jintal » 28 Jan 2009, 20:45

Running a JVM on an SPE.. interesting!

Thanks for the spu-top tip, was just about to research on how to view those processes :D

Guess i'm going for writing C code for the Cell Processor. How much limited are the SPEs? I know the their memory is small, and they're there to perform vector computations.. but can they at least read / write local text files?
jintal
ydl newbie
ydl newbie
 
Posts: 11
Joined: 23 Jan 2009, 12:55

Re: Does Mysql / Python Benefit From The Cell Processor?

Postby ppietro » 28 Jan 2009, 22:51

jintal wrote:Running a JVM on an SPE.. interesting!

Thanks for the spu-top tip, was just about to research on how to view those processes :D

Guess i'm going for writing C code for the Cell Processor. How much limited are the SPEs? I know the their memory is small, and they're there to perform vector computations.. but can they at least read / write local text files?


Hmmm - I'm not sure if they support File I/O or not. I know they can write to main memory or be polled by the PPE. But writing to disc? It might be beyond their capabilities. I think it's best to consider the SPEs like programmable numeric co-processors more than anything else. You'd probably want to consider a sender/receiver model - like a thread handler.

That's about the limit of my knowledge on this. This is one of those times I'm going to have to defer to an actual Cell programmer for follow-up. Sorry.

Cheers,
Paul
User avatar
ppietro
Site Admin
Site Admin
 
Posts: 4965
Joined: 13 Sep 2007, 22:18

Re: Does Mysql / Python Benefit From The Cell Processor?

Postby jintal » 29 Jan 2009, 21:59

Thanks guys, you've all been a great help!

I'm going to test if the SPE is capable of performing file IO in a day or two. For the sake of this thread, i'll post my finding here :D
jintal
ydl newbie
ydl newbie
 
Posts: 11
Joined: 23 Jan 2009, 12:55

Re: Does Mysql / Python Benefit From The Cell Processor?

Postby ppietro » 29 Jan 2009, 22:34

jintal wrote:Thanks guys, you've all been a great help!

I'm going to test if the SPE is capable of performing file IO in a day or two. For the sake of this thread, i'll post my finding here :D



This page should help:
http://linux.die.net/man/2/spufs

Cheers,
Paul
User avatar
ppietro
Site Admin
Site Admin
 
Posts: 4965
Joined: 13 Sep 2007, 22:18

Re: Does Mysql / Python Benefit From The Cell Processor?

Postby jintal » 01 Feb 2009, 17:18

Hi again!

I successfully compiled and ran a small script that uses an SPE to read the contents of a file and print them out. The SPU process created is shown when i run spu-top and spu-ps.

I've created 2 compiled scripts, an SPE one, then a PPU one. Running the PPU script would create an SPE thread which in turn reads and prints the contents of a file.

I have a question again though.. does that mean that the whole SPE process uses purely the SPE when it does its file reads? Or does the SPE call on the PPU to do the file reading and/or the printing of the contents? Is there some command which can validate this?

FYI, in my tests, running the command top shows the PPU app only, while running spu-top shows the correct SPE processes.

-edit-
I looked more closely at the spu-top display.
When running the script that does only SPE computations, i get a CPU activity of 3%, and an SPU activity of 100%.
When running the script that does SPE file reads, i get a CPU activity of 80% and an SPU activity of 3%.

Does this mean that ideally SPEs should only do internal computations?
jintal
ydl newbie
ydl newbie
 
Posts: 11
Joined: 23 Jan 2009, 12:55

Re: Does Mysql / Python Benefit From The Cell Processor?

Postby ppietro » 02 Feb 2009, 05:39

jintal wrote:Does this mean that ideally SPEs should only do internal computations?


That is my understanding, yes. They can handle writes to and from main memory via DMA, but as for other I/O (like file reading/writing), I have no idea. My guess is that they would have to filter thru the PPE.

Cheers,
Paul
User avatar
ppietro
Site Admin
Site Admin
 
Posts: 4965
Joined: 13 Sep 2007, 22:18

Previous

Return to Software Development

Who is online

Users browsing this forum: No registered users and 2 guests