AMD Fusion and GPGPU killer app

Page 2 - Seeking answers? Join the AnandTech community: where nearly half-a-million members share solutions and discuss the latest tech.

piesquared

Golden Member
Oct 16, 2006
1,651
473
136
You'll likely be one of the few if WinZip outperforms it by a conciderable amount.

Another great app will be Steady Video 2.0 built on the original which showed tangible benefits.
 

Arkaign

Lifer
Oct 27, 2006
20,736
1,377
126
Honestly I expected a much better app than Winzip when clicking the thread title, hah.

I think it could be promising, but would have to start affecting a whole lot more than just Winzip to be relevant.

And that brings us to another problem. Wouldn't you still have the same (or perhaps better?) benefits by using an Intel CPU along with an AMD GPU? As that is the case with a ton of PC enthusiasts who are more likely to care about something like this. I'm not sure joe 6-pack buying a $299 Dell at Walmart is going to know or care that the AMD box will unpack his 30mb zip file in 2 seconds instead of 2.8 on the Intel box
 

podspi

Golden Member
Jan 11, 2011
1,982
102
106
I think the use could be more substantial than you'd think. For example, when Intel added AES I thought that was a waste of silicon, because I had never needed *that* much encryption (and hey, today's CPUs are fast enough anyways!). Now, we have disk-wide encryption.


I could defintely make the argument for disk-wide compression as well. For example, I have one folder that is 630MB (work-related stuff), that compresses down to 42MB with WinRAR. Now I realize many things won't compress that well, but imagine if we could start saving everything in lossless uncompressed formats, with the compression happening in real-time? That would be cool beans...
 

piesquared

Golden Member
Oct 16, 2006
1,651
473
136
I think the use could be more substantial than you'd think. For example, when Intel added AES I thought that was a waste of silicon, because I had never needed *that* much encryption (and hey, today's CPUs are fast enough anyways!). Now, we have disk-wide encryption.


I could defintely make the argument for disk-wide compression as well. For example, I have one folder that is 630MB (work-related stuff), that compresses down to 42MB with WinRAR. Now I realize many things won't compress that well, but imagine if we could start saving everything in lossless uncompressed formats, with the compression happening in real-time? That would be cool beans...

And because programs like this are using OpenCL, it's scalable with the GPU. So right down to the low end markets where they can still pack a lot of cores for acceleration, AMD would have a fairly big advantage given their lead in GPGPU over intel. Anyway, it won't take long to see what happens, it's playing out as we speak!
 

Kenmitch

Diamond Member
Oct 10, 1999
8,505
2,249
136
I could defintely make the argument for disk-wide compression as well. For example, I have one folder that is 630MB (work-related stuff), that compresses down to 42MB with WinRAR. Now I realize many things won't compress that well, but imagine if we could start saving everything in lossless uncompressed formats, with the compression happening in real-time? That would be cool beans...
 


SSD makers would rejoice as sales climbed.
 

Vesku

Diamond Member
Aug 25, 2005
3,743
28
86
I believe Sandforce SSDs compress the data as part of the disk writing. Would be interesting if software based SSD as cache incorporated a compression option.
 

BrightCandle

Diamond Member
Mar 15, 2007
4,762
0
76
Imagine a world where your entire hard drive is compressed, but the only impact on performance is that it increases it because you can decompress faster with your GPU than you can read and write to the disk. Apart from Sandforce drives this would improve performance immensely. That is a killer application of GPGPU.

My personal opinion is that openCL isn't likely to be the solution long term. Its a pain in the bum to program and too GPU centric in its design. A shared memory space x86 compatible solution with proper multitasking is what is needed, ideally something as simple as just setting a thread to be possible on the GPU. That way the OS schedules the GPU cores like the CPU ones and all the programmer does is designate a thread as benefiting. Maybe in the future the OS wont need a hint and will only use the GPU threads when it runs out of CPU ones, or via heuristic recognises lots of vector instructions. AMD is putting all the necessary logic onto their GPU cores with GCN for much of this vision.
 

Idontcare

Elite Member
Oct 10, 1999
21,118
59
91
Imagine a world where your entire hard drive is compressed, but the only impact on performance is that it increases it because you can decompress faster with your GPU than you can read and write to the disk. Apart from Sandforce drives this would improve performance immensely. That is a killer application of GPGPU.

Imagine such a world?

I don't have to imagine it, just about anyone that lived through the age of 386 based computers likely had both their harddrive as well as their ram "doubled" by such software compression tricks.

It was a gimmick, sure you got more effective data space but the performance hit will always be there. The only reason to do it is if the cost of the underlying hardware is prohibitive relative to the expense of purchasing the computational resources (CPU) necessary to minimize the latency of the compression/decompression middleware.

There's a reason its not used on supercomputers. No ram doublers there.

It would be cool if dedicated compress/decompress circuits were built into all data IO's (HDD as well as your ram) very much how your old 14.4 modems worked to boost speed over POTS.

But you really don't want to use general purpose compute circuits for that anymore than you want your Areca 1882 to drop the dual-core 0.8GHz onboard processor and bootstrap onto your x86 CPU, or say to use your x86 CPU to emulate your video processor
 

sm625

Diamond Member
May 6, 2011
8,172
137
106
Now I realize many things won't compress that well, but imagine if we could start saving everything in lossless uncompressed formats, with the compression happening in real-time? That would be cool beans...

It's called SandForce.
 

podspi

Golden Member
Jan 11, 2011
1,982
102
106
I know that the SandForce controller uses compression to increase write speeds, but I don't see anything about it using it to increase the total amount of storage space available. I understand these go hand-in-hand, but does this mean I could store terabytes of 0-filled textfiles on a SandForce-based SSD, or does it mean it would just save said file quickly?



Anyway, my point wasn't that full-disk compression is the killer-app of the future (again, as IDC pointed out), but that once the capability is there people will find uses for things...
 

Vesku

Diamond Member
Aug 25, 2005
3,743
28
86
SandForce's compression scheme is proprietary so not too much detailed information is available. But my understanding is it is for speed AND to enhance write lifespan, it can store the info in less flash cells while still providing the customer with their rated capacity. I cited SandForce as an example of dedicated compression circuitry in action and how the same thing could be done in software with GPGPU and thus not bog down a CPU. Just as CPUs have added dedicated encryption logic to speed up full disk encryption we might see GPUs used for full disk compression.

If I was finding solutions for full encryption and compression it seems like APUs will provide decent options in a generation or two.
 
Last edited:

drizek

Golden Member
Jul 7, 2005
1,410
0
71
I know that the SandForce controller uses compression to increase write speeds, but I don't see anything about it using it to increase the total amount of storage space available. I understand these go hand-in-hand, but does this mean I could store terabytes of 0-filled textfiles on a SandForce-based SSD, or does it mean it would just save said file quickly?



Anyway, my point wasn't that full-disk compression is the killer-app of the future (again, as IDC pointed out), but that once the capability is there people will find uses for things...

It reduces the need for spare area, so yes, it does increase the effective storage capacity.
 

Chiropteran

Diamond Member
Nov 14, 2003
9,811
110
106
Imagine such a world?

I don't have to imagine it, just about anyone that lived through the age of 386 based computers likely had both their harddrive as well as their ram "doubled" by such software compression tricks.

I remember that too. But it's really just a mathematical equation. Time to compress/decompress=X, time to read from drive=Y.

If X is less than (Y/2) and compression rate is 50% or better on average, the reduction in time spent reading from the hard drive would overcome the extra time spent processing. And if the processing is done using an otherwise idle GPU, it's not taking away processing time from any other applications.

I see it as a possibility of actually adding real performance, if done correctly.
 

Phynaz

Lifer
Mar 13, 2006
10,140
819
126
If X is less than (Y/2) and compression rate is 50% or better on average, the reduction in time spent reading from the hard drive would overcome the extra time spent processing. And if the processing is done using an otherwise idle GPU, it's not taking away processing time from any other applications.

I see it as a possibility of actually adding real performance, if done correctly.

And then you hit incompressible data, and your performance goes down the tubes.

It's been done before, by many companies, including Microsoft (it was a part of the OS at one time).
 

Vesku

Diamond Member
Aug 25, 2005
3,743
28
86
If it's using the gpu portion of a chip that would otherwise go underutilized then it's no longer a tradeoff.
 

Chiropteran

Diamond Member
Nov 14, 2003
9,811
110
106
And then you hit incompressible data, and your performance goes down the tubes.

Why would performance go down at all? The code could be smart enough that if you can't compress at least by 25% (or some other value) it just flags it as uncompressed and skips processing on that file. No compression work, no performance cost.
 

Phynaz

Lifer
Mar 13, 2006
10,140
819
126
Why would performance go down at all? The code could be smart enough that if you can't compress at least by 25% (or some other value) it just flags it as uncompressed and skips processing on that file. No compression work, no performance cost.

In order to determine if the data is compressible, you practically have to compress it.

Really, you're not getting it. It's been tried before. The same argument was used before. It doesn't pay. The overhead is too large.

The only place it's used today is in WAN links.
 
Last edited:

micrometers

Diamond Member
Nov 14, 2010
3,473
0
0
I imagine that there is a man out there who has devoted the past ten years of his life to developing Winzip.
 

Cerb

Elite Member
Aug 26, 2000
17,484
33
86
Don't see the killer app in this. It' s not like it takes hours to unzip "small" files and in the end that it was 99.9% of users do use it for.
With any CPU from the last 6+ years, it's not slow enough zipping files to worry about the speed, either. The algorithm as normally implemented is already very fast.

Now, if you could get LZMA2 or PBZIP2 GPU-accelerated, that'd be something. I imagine PBZIP2 should be possible, though not easy, to do.

hard to imagine CUDA winning even if amd didn't existed.
It won't. For all AMD's bark, which I hoped they'd follow up on, NVidia and Intel have had the bite. NVidia is keeping CUDA cutting-edge, for their hardware, and supporting other standards as well as anybody else, if not better. They would be in fine shape tomorrow (well, at least in GPGPU software terms) if everyone abruptly decided to not start any new CUDA projects.

I think the use could be more substantial than you'd think. For example, when Intel added AES I thought that was a waste of silicon, because I had never needed *that* much encryption (and hey, today's CPUs are fast enough anyways!). Now, we have disk-wide encryption.
It saves power on notebooks for users that need it, and is quite useful for file servers.

I could defintely make the argument for disk-wide compression as well. For example, I have one folder that is 630MB (work-related stuff), that compresses down to 42MB with WinRAR. Now I realize many things won't compress that well, but imagine if we could start saving everything in lossless uncompressed formats, with the compression happening in real-time? That would be cool beans...
I've been doing that since 1997 (NT4--the same as saying 'since 2011' for something starting today ), when my largest HDD was 8GB. Today, I compress directories of games with mods, as they tend not to be neatly packaged in single large files, so NTFS compression helps quite a bit, for my mechanical drive.

With faster storage, however, the overhead outweighs the gains, bring it back to a pure space issue, where keeping unchanging data in archives works as well today as it did back when storage was expensive.

Getting the kind of compression you get with documents that you know are similar to each other, for the OS, FS, or drive, would need multi-pass statistical analysis, which would be best started at the time of formatting (1A. approximate compressibility of data 1B. approximate similarities amongst file contents 2. create a database of analyzed 1A and 1B data, then differentially compress, dedup, sort, and compress results as needed, using that database to more efficiently compress any new files).

What grouping appears easy to you at a high level is not so easy to implement effectively at a much lower level. It can be done, but not without a price. Going by filetype, FI, would leave you with either longer compression times as that amount of data increased, or reduced compression if done in small chunks, or the need for additional FS metadata processing overhead to find what small set of files it should be compressed along with, which would still result in lesser compression next to your choice of a single high-compression archive, and often necessitate several additional reads per file write (also, some formats these days compress themselves individually, which would result in poor compression after the fact).

It can be done, but the demand is lacking for reasons of cheap storage, performance, and that big stuff people would like smaller tends to already be compressed.
 
Last edited:

WhoBeDaPlaya

Diamond Member
Sep 15, 2000
7,414
401
126
Imagine a world where your entire hard drive is compressed, but the only impact on performance is that it increases it because you can decompress faster with your GPU than you can read and write to the disk.
Your age is showing, and so is mine.
Been there, done that back in the day with MS-DOS 6.0 on a 286 with a 40MB HDD.
Took bloody forever for the initial compression
 

Chiropteran

Diamond Member
Nov 14, 2003
9,811
110
106
In order to determine if the data is compressible, you practically have to compress it.

Really, you're not getting it. It's been tried before. The same argument was used before. It doesn't pay. The overhead is too large.

The only place it's used today is in WAN links.

I get it, but maybe you don't.

The part where performance matters is reading off the hard disk, because writes can simply be buffered indefinitely. When you are READING from the hard disk, the data is either already compressed or not, so there is no extra work determining if it is compressible or not. When you are WRITING to the hard disk, as far as any applications are concerned the write is instantaneous even if the write isn't made to the disk yet. Only later, when spare cycles are available does the actual hard disk write need to occur, and it doesn't matter if the compression makes this take longer.
 

ncalipari

Senior member
Apr 1, 2009
255
0
0
If Intel supports openCL, I don't see how CUDA could win in the long run. Wouldn't developers much rather code for something that runs on Intel, AMD and Nvidia's chips?

CUDA aims ultimately at a different sector.

If you are a company, and want to develop a custom software to be accelerated with GPGPU, then CUDA is the best choice, as Nvidia GPUs are better performing in most real world cases and easier to program.

http://fastra2.ua.ac.be/


Anyhow if you aim at final users, then OpenCL is the way to go.
 

ncalipari

Senior member
Apr 1, 2009
255
0
0
Really, you're not getting it. It's been tried before. The same argument was used before. It doesn't pay. The overhead is too large.


It's never been tried before with as much computational power as GPGPU have. We're using a GPGPU-assisted Database compression software with EXTREMELY good results.
 
sale-70-410-exam    | Exam-200-125-pdf    | we-sale-70-410-exam    | hot-sale-70-410-exam    | Latest-exam-700-603-Dumps    | Dumps-98-363-exams-date    | Certs-200-125-date    | Dumps-300-075-exams-date    | hot-sale-book-C8010-726-book    | Hot-Sale-200-310-Exam    | Exam-Description-200-310-dumps?    | hot-sale-book-200-125-book    | Latest-Updated-300-209-Exam    | Dumps-210-260-exams-date    | Download-200-125-Exam-PDF    | Exam-Description-300-101-dumps    | Certs-300-101-date    | Hot-Sale-300-075-Exam    | Latest-exam-200-125-Dumps    | Exam-Description-200-125-dumps    | Latest-Updated-300-075-Exam    | hot-sale-book-210-260-book    | Dumps-200-901-exams-date    | Certs-200-901-date    | Latest-exam-1Z0-062-Dumps    | Hot-Sale-1Z0-062-Exam    | Certs-CSSLP-date    | 100%-Pass-70-383-Exams    | Latest-JN0-360-real-exam-questions    | 100%-Pass-4A0-100-Real-Exam-Questions    | Dumps-300-135-exams-date    | Passed-200-105-Tech-Exams    | Latest-Updated-200-310-Exam    | Download-300-070-Exam-PDF    | Hot-Sale-JN0-360-Exam    | 100%-Pass-JN0-360-Exams    | 100%-Pass-JN0-360-Real-Exam-Questions    | Dumps-JN0-360-exams-date    | Exam-Description-1Z0-876-dumps    | Latest-exam-1Z0-876-Dumps    | Dumps-HPE0-Y53-exams-date    | 2017-Latest-HPE0-Y53-Exam    | 100%-Pass-HPE0-Y53-Real-Exam-Questions    | Pass-4A0-100-Exam    | Latest-4A0-100-Questions    | Dumps-98-365-exams-date    | 2017-Latest-98-365-Exam    | 100%-Pass-VCS-254-Exams    | 2017-Latest-VCS-273-Exam    | Dumps-200-355-exams-date    | 2017-Latest-300-320-Exam    | Pass-300-101-Exam    | 100%-Pass-300-115-Exams    |
http://www.portvapes.co.uk/    | http://www.portvapes.co.uk/    |