Yellow Dog Linux

Posted: **23 Jan 2009, 13:57**

Hi all,

With YDL in a Playstation 3, do MySQL and/or Python use the cell processor fully?

If MySQL is compiled in YDL in a PS3, does is spread it's processes across the SPEs?

With Python, does the same apply? or do I need to get my hands dirty with CorePy?

Hope anyone can give me an idea on this, i've searched been searching the web and forums and couldn't find a definitive answer.

Thanks!

Posted: **23 Jan 2009, 15:02**

No, neither of those use the Cell's SPEs unless code is written/modified to take advantage of them -- simply compiling them on YDL won't make any difference. I've looked at CorePy but quickly decided it was either above my head or too early in development, or both.

(see here).

Posted: **23 Jan 2009, 16:45**

How about using C/C++ then compiling with YDL's gcc? or RapidMind?

Posted: **23 Jan 2009, 18:35**

jintal wrote:How about using C/C++ then compiling with YDL's gcc? or RapidMind?

No to either of those, as far as I'm aware. I'm not familiar with RapidMind, but I do know that with C/C++, you have to code manually for the SPEs.

Remember that the SPEs in the Cell do not execute PowerPC code directly - it's not a case of setting a "Cell" target and having them be used. They have their own instruction set - and their own memory for code. Basically, the PPE in the Cell is used in a manner similar to an orchestra conductor - it sends pre-compiled SPE-specific code to each SPE, starts them executing, then monitors their status.

Generally speaking, you have to re-architect any program specifically to take advantage of the SPEs in a Cell.

You might start looking here:
http://us.fixstars.com/showcase/cellebration/mit/

Cheers,
Paul

Posted: **24 Jan 2009, 20:22**

Hi,

I've been thinking about your replies and researched more.

Am I correct in saying that (currently) the only way to fully utilize the cell processor is by programming in C alongside the IBM Cell SDK?

Posted: **24 Jan 2009, 21:42**

jintal wrote:Am I correct in saying that (currently) the only way to fully utilize the cell processor is by programming in C alongside the IBM Cell SDK?

That sounds about right - yes. Due to memory limitations, most folks program the SPUs in Assembly Language, but there are C/C++ compilers available.

This looks like a good introduction:
http://www.ibm.com/developerworks/libra ... inuxps3-5/

Bear in mind that the SPU does not contain a full "Turing Compliant" instruction set. It's primarily used for math functions - it's especially tuned for vector math operation.

Cheers,
Paul

Posted: **24 Jan 2009, 23:13**

ppietro wrote:...Due to memory limitations, most folks program the SPUs in Assembly Language, but there are C/C++ compilers available.

Are you talking about lipspe2? It seems (I think) that this is a C library used to compile PPE and SPE specific C programs. I have YDL 6.1 on my PS3, and it looks like it's already included in the default installation.

The link you gave helps a lot in understanding (only absorbed a bit so far) the PPEs and the SPEs and other techie terms. Although I noticed that the article is quite old and uses a previous version of the SDK, I couldn't test out some stuff in the actual environment.

Would these sources also help? I'm new to this Cell Processing and I'm not fluent in C.

http://www.ibm.com/developerworks/library/pa-libspe2/
http://www.lemma.ufpr.br/wiki/index.php/Cell_BE_Tutorial

Feels like I've read tons of articles already, although I'm still not sure about anyting..

The reason why I bought this PS3 was to install Linux to perform some calculations. Looks like there's a long road ahead for me..

Posted: **25 Jan 2009, 04:44**

jintal wrote:Are you talking about lipspe2? It seems (I think) that this is a C library used to compile PPE and SPE specific C programs. I have YDL 6.1 on my PS3, and it looks like it's already included in the default installation.

lipspe2? You mean libspe2?

Actually, according to this:
http://www.ibm.com/developerworks/library/pa-libspe2/

it's just an interface library of functions to allow the PPU to control the SPUs. It isn't a compile library per se.

jintal wrote:Would these sources also help? I'm new to this Cell Processing and I'm not fluent in C.
http://www.ibm.com/developerworks/library/pa-libspe2/
http://www.lemma.ufpr.br/wiki/index.php/Cell_BE_Tutorial

They couldn't hurt. I referenced your libspe2 link above.

Just FYI - I primarily program in Java, which doesn't take advantage of the Cell architecture either. So - I'm probably going to bow out of this conversation and let actual PPU/SPU programmers respond. So far, nothing I've needed to program has required Cell specific optimization.

Cheers,
Paul

Posted: **25 Jan 2009, 08:12**

ppietro wrote:lipspe2? You mean libspe2?

Oops! It was late and I was tired from figuring this thing out

Yup, i mean libspe2

Ah you use java with the ps3? Are you doing it there to get some speed benefits or just because?

I wrote a simple python script to test the speed of the execution, in my core 2 duo, the script ran for 6 secs. In YDL in PS3, it ran for 20 secs.. I guess that's the PPE doing all the job.

Posted: **25 Jan 2009, 10:28**

jintal wrote:Ah you use java with the ps3? Are you doing it there to get some speed benefits or just because?

Just because mostly. I'm not a big fan of C or C++, to be honest. Couldn't tell you why exactly - they just seem to rub me the wrong way. I guess it's stuff like pointers, handles, and the "feature" in C that allows you to reduce extremely complex C code to a couple of incomprehensible lines. Java's kinda like a C that doesn't let you "cheat", I guess.

And - I use Java in general - not just PS3. Of course, the side benefit is that any Java I write anywhere runs on the PS3 without a recompile. That's another reason I dig Java. That - and the IBM Java for the PS3 is pretty decent.

BTW - that's the first thing I do when I perform a fresh install of YDL. I replace the open source GCJ with IBM's PowerPC Java. Much faster and lighter.

jintal wrote:I wrote a simple python script to test the speed of the execution, in my core 2 duo, the script ran for 6 secs. In YDL in PS3, it ran for 20 secs.. I guess that's the PPE doing all the job.

Exactly - no SPEs were invoked. Don't expect wonders from the PPE. It's a single core, dual threaded, moderately fast, in-order execution PowerPC processor. One cool thing about it, though, is that it does have the AltiVec extension. I wasn't actually expecting that.

Cheers,
Paul

Posted: **26 Jan 2009, 01:21**

ppietro wrote:That - and the IBM Java for the PS3 is pretty decent. BTW - that's the first thing I do when I perform a fresh install of YDL. I replace the open source GCJ with IBM's PowerPC Java.

Which I keep forgetting to get around to doing.

Exactly - no SPEs were invoked. Don't expect wonders from the PPE. It's a single core, dual threaded, moderately fast, in-order execution PowerPC processor. One cool thing about it, though, is that it does have the AltiVec extension. I wasn't actually expecting that.

Cheers,
Paul

And there are applications that use Altivec, GIMP does. ffmpeg does if I've read correctly

Ron Rogers Jr. (CronoCloud)

Posted: **26 Jan 2009, 01:57**

CronoCloud wrote:And there are applications that use Altivec, GIMP does. ffmpeg does if I've read correctly

ffmpeg doesn't per se - but some of its libraries do. x264 being an example:

[paulp@yurie daily_show]$ ffmpeg -i ./20080403_230000_71.mpg -vcodec h264 -acodec aac outtest.mp4
FFmpeg version SVN-r8876, Copyright (c) 2000-2007 Fabrice Bellard, et al.
configuration: --prefix=/usr --incdir=/usr/include/ffmpeg --libdir=/usr/lib --shlibdir=/usr/lib --mandir=/usr/share/man --arch=powerpc --extra-cflags=-O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector --param=ssp-buffer-size=4 -m32 --enable-libmp3lame --enable-libogg --enable-libtheora --enable-libvorbis --enable-libfaad --enable-libfaac --enable-libgsm --enable-xvid --enable-x264 --enable-liba52 --enable-liba52bin --enable-libdts --enable-pp --enable-pthreads --disable-static --enable-shared --enable-gpl --disable-debug --disable-opts --disable-strip
libavutil version: 49.4.0
libavcodec version: 51.40.4

8< snip >8

Output #0, mp4, to 'outtest.mp4':
Stream #0.0: Video: h264, yuv420p, 640x480, q=2-31, 200 kb/s, 30.00 fps(c)
Stream #0.1: Audio: aac, 44100 Hz, stereo, 64 kb/s
Stream mapping:
Stream #0.0 -> #0.0
Stream #0.1 -> #0.1
[h264 @ 0xf1ed8b4]using SAR=1/1
[h264 @ 0xf1ed8b4]using cpu capabilities Altivec
Press [q] to stop encoding
frame= 12 fps= 3 q=2248.0 Lsize= 67kB time=0.4 bitrate=1371.2kbits/s

Cheers,
Paul

Posted: **27 Jan 2009, 05:53**

How does one use this AltiVec?

ppietro wrote:I'm not a big fan of C or C++, to be honest

Same here!

ppietro wrote:Couldn't tell you why exactly - they just seem to rub me the wrong way.

Same here too!

That's why I was wondering if MySQL and/or Python has some special implementation just for Cell Processors (like GIMP does according to CronoCloud)

Posted: **27 Jan 2009, 06:07**

jintal wrote:That's why I was wondering if MySQL and/or Python has some special implementation just for Cell Processors (like GIMP does according to CronoCloud)

Just to be clear here - the GIMP is not Cell optimized - it's PowerPC optimized. The AltiVec is not Cell specific - it's been on a number of PowerPC processors, which includes the PPE of the Cell.

There's a good description here:
http://en.wikipedia.org/wiki/Altivec

Cheers,
Paul

Posted: **27 Jan 2009, 06:25**

Oh, It's because it came with the AltiVec. :oops:

Thanks for clearing that up!

How about this? http://www.linuxjournal.com/article/8497
I'm thinking... write short PPE and SPE C code, wherein the SPE one has the Python interpreter embedded in it. Any good?

Yellow Dog Linux

Does Mysql / Python Benefit From The Cell Processor?

Does Mysql / Python Benefit From The Cell Processor?

Re: Does Mysql / Python Benefit From The Cell Processor?

Re: Does Mysql / Python Benefit From The Cell Processor?

Re: Does Mysql / Python Benefit From The Cell Processor?

Re: Does Mysql / Python Benefit From The Cell Processor?

Re: Does Mysql / Python Benefit From The Cell Processor?

Re: Does Mysql / Python Benefit From The Cell Processor?

Re: Does Mysql / Python Benefit From The Cell Processor?

Re: Does Mysql / Python Benefit From The Cell Processor?

Re: Does Mysql / Python Benefit From The Cell Processor?

Re: Does Mysql / Python Benefit From The Cell Processor?

Re: Does Mysql / Python Benefit From The Cell Processor?

Re: Does Mysql / Python Benefit From The Cell Processor?

Re: Does Mysql / Python Benefit From The Cell Processor?

Re: Does Mysql / Python Benefit From The Cell Processor?