Beta
×

Welcome to the Slashdot Beta site -- learn more here. Use the link in the footer or click here to return to the Classic version of Slashdot.

Thank you!

Before you choose to head back to the Classic look of the site, we'd appreciate it if you share your thoughts on the Beta; your feedback is what drives our ongoing development.

Beta is different and we value you taking the time to try it out. Please take a look at the changes we've made in Beta and  learn more about it. Thanks for reading, and for making the site better!

Comments

top

Back To Faxes: Doctors Can't Exchange Digital Medical Records

KWTm Agree: doctors fall into EMR vendor lock-in trap (240 comments)

As a physician who was dragged screaming and kicking into having to use EPIC, I have to agree.

I never knew I could hate a company more than Microsoft. Their client is a bloated horror that nevertheless acts like the thinnest client in the world: "Oh, look, the doctor pressed the Shift key ... I guess I'll send that over the network, and wait for a response ... oh look, s/he released the Shift key now -- I guess I'll send that over the network, too..." Apparently it's based on the Internet Explorer library, so there is no Mac version (at least not when I was using it)...

The interface was so bad that I learned how to program in AutoHotkey and probably spent in excess of 200 hours over a year to automate things. AutoHotkey was a lifesaver: open source and powerful. (In fact, the pitiful xdotool we have for Linux doesn't even come close to AutoHotkey for windows, and even if I weren't forced to use Windows for my work, I might have ended up choosing it over Linux just because of AutoHotkey and its ecosystem of experienced developers.)

At the time I was with a large clinic chain that had about 40% of the market in our large sprawling metropolitan supercluster location. They surveyed the doctors, who said that on average they were spending an extra hour per day using Epic. And in the end, it was a lot of *data*-generation and not a lot of *information*. Our specialists complained that everything was being crammed into a template form, and they really couldn't tell what we were thinking, just checklists of what the patient did/did not have.

Having vendor lock-in, they have no incentive to improve. They can do whatever they want... if the clinic/hospital is already stuck using Epic, why would they spend money on fixing their problems instead of recruiting more clients?

Having said all that, even Epic is better than what I'm stuck using right now ... eClinicalWorks. That's even worse than Epic. All the problems of Epic, plus even worse interface. Right now I type my notes in a plain text editor and then use AutoHotkey to cut-n-paste it into eClinicalWorks. What a nightmare.

OpenEMR all the way!

about three weeks ago
top

Help EFF Test a New Tool To Stop Creepy Online Tracking

KWTm CookieController deletes cookies with 1click (219 comments)

I use Cookie Controller. Among other things, it has a handy button to click on. On the first click, it will wipe out temporary (session) cookies for the site you're on right now. On the second click, it will wipe persistent cookies, too. The third click wipes out session cookies for all sites. A fourth click will wipe all cookies. The button appearance changes to let you know what it's going to do, and in case you forget, hovering over the button brings up a tooltip that tells you what sorts of cookies and how many are about to get wiped.

Very handy now that Google is tracking everything. I don't particularly want all my casual searches to be linked to my Google maps requests and my Google translates.

The plugin doesn't sound as automated as Self-Destructing Cookies, so maybe I will check it out.

about 5 months ago
top

Why San Francisco Is the New Renaissance Florence

KWTm Palo Alto is Spanish for "perpetual traffic jam" (250 comments)

A benefit with Palo Alto and surrounding communities is that you can actually find parking.

Yes, traffic flows so slowly through Palo Alto (including Highway 101 on weeknights where drivers slow to a crawl as soon as you enter Palo Alto) that you can always find parking. You see, the entire city of Palo Alto is one big parking lot!

about 7 months ago
top

It's Not Memory Loss - Older Minds May Just Be Fuller of Information

KWTm Agree: So, can computer RAM literally fill up? (206 comments)

Agree. Unless "fill up" is interpreted this way, you might similarly say of the claim that "my computer RAM has literally filled up and there are zero bytes free" that there has been no physical cavity within the RAM chips which have decreased in volume due to contents physically occupying volume.

about 9 months ago
top

Ask Slashdot: Life After N900?

KWTm also: resistive screen is okay if it's multi-touch (303 comments)

I hear a lot of people disparaging the resistive touch-screen of the N900, compared to the Technically Awe-Inspiring And Can't-Be-Topped (or "Apple" for short) capacitive touchscreen.

For the record, I think people just prefer the multi-touch capability of the capacitive screen. If the resistive screen could be multi-touch, then it would be okay (and much more high-res, apparently).

Well, Neo900 hardware is going to support dual-touch gestures like rotating and pinching, without replacing the original, resistive screen from N900! So you can have dual-touch on resistive. I think that addresses the complaint about resistive. Otherwise, what exactly is the advantage of a capacitive touchscreen?

about 9 months ago
top

Ask Slashdot: Life After N900?

KWTm Scripting on the Android? Don't make me laugh (303 comments)

important that I run Vim.

Looking at the N900 keyboard I don't think Vim would be very usable...

Not sure what you mean. I actually and currently *do* use vim on my N900. With the keyboard.
It's what I use to take notes.
And also make phone calls (a keystroke mapping makes vim read the current line, identify the phone number on it, and call that number).
And also send text messages (a keystroke mapping makes vim identify the phone number at the start of the line, and send the rest of the line to that number as a SMS).
And also read incoming text messages -- handy when I'm driving and can't squint at the screen (a keystroke mapping makes vim call a bash script that pulls the latest text message via SQLite3, and then read it aloud using espeak).
And a similar keystroke automatically sends a reply to whoever sent the SMS, telling him/her that I got the message, but I'm driving and can't respond right now.
And so on, for "silence the N900 for (specified time, default 1 hour) then re-enable sound" because I'm in a meeting, etc. etc.
In fact, I mostly control my N900 from the vim screen, and sometimes from the bash screen; much more fine-grained control than the GUI interface. The hardware keyboard is what makes it usable when driving; with an onscreen keyboard, you couldn't keep your eyes on the road. (Cue the snarky comments about "You should never use your phone when driving, because I don't, and everyone else should be like me.")

Oh, part of the reason Vim works well on my N900 keyboard is because I've redefined it. By defaut each key has a meaning when pressed by itself, or with the Shift key, or with the Fn key. But there's a fourth combination of Shift+Fn key, which by default is the same as the Fn key -- which is completely wasted potential. So, by using the Shift+Fn key combo, I've got 30 extra characters to use, including []{}`|^~% which by default you'd have to call up the virtual keyboard to type. Maybe that's why you think the keyboard is not usable with Vim.

*I* want to be the one in control of the phone

There are lots of automation/scripting apps for Android, or with root you can get a real shell and install the scripting language of your choice.

I'd be interested in a "real" shell/scripting language. Right now I have a bash script for synchronizing various text files (such as my ToDo list) between various laptops/desktops, and I just use the same script for my N900. The script also synchronizes itself, so when I improve the script on my work laptop, for example, the versions on the other laptop and the desktop and, yes, my N900 will update themselves.

I see some really simple tools on Android -- a brief web search produces results like Tasker and AutomateIt, which are great when you want simple things like, say, changing the ringtones every Wednesday afternoon unless your phone is face-down.

I'm looking for things more like, if a SMS comes in, then make a different "SMS arrived" sound depending on who it's from; and if it's from the wife or my parents, then read the SMS out loud unless I'm at my workplace; otherwise forward the SMS contents to my email account. Here's a N900 script for "while I'm at the office, adjust my Google Voice virtual phone settings to not forward phone calls to home". Right now I'm working on something that sends a SMS to my wife telling what freeway exit I'm closest too, and my driving speed, so I don't have to manually text her while driving to let her know when I'll be home.

Tasker falls short in these abilities, to put it mildly; never mind that you can't reuse the scripts for the desktop/laptop. I don't mean to belittle your helpful info; it's just that smartphones like Android devices just aren't in the same league as computers-that-happen-to-have-phones-built-in like the N900. At the same time, I recognize that currently there's nothing quite like the N900 on the market.

So, the other choice that you mention is "with root you can get a real shell". I guess by rooting you mean jailbreaking. CyanogenMod seems to offer a lot of freedom compared to the as-is Android device, although it's a lot easier to achieve on the N900, and without jailbreaking.

How good is the hardware control in CyanogenMod-based scripting? Can you write a bash (or python) script for, say, connecting to a certain wifi? Turning phone off/on? Turning GPS on at pre-programmed intervals, getting a location fix, and then turning off again? If it's capable of that, then I'm going to keep CyanogenMod as my backup plan if nothing feasible comes out by the time my N900's fail.

Thanks for any info you (or anyone else) can provide about CyanogenMod scripting.

about 9 months ago
top

$499 3-D Printer Drew Plenty of Attention at CES (Video)

KWTm Brother MFC-665cw keeps ink from clogging (155 comments)

If unusued for a week or two, my Brother printer will exercise the ink cartridge briefly to prevent it from blocking. Yes, this does involve using up a bit of the ink. I had our brother printer sit unusued for a year, plugged in, turned on (just in case we needed it -- but we happened to be in a phase of our lives where we didn't for a long time). I noticed that the ink level would go down slowly even when unused.

Compare that to my old HP inkjet. Sat unusued for a season or so, and then the ink cartridges were unusable. Had to be replaced.

Don't know if exercising the cartridge was the only factor, but I am impressed by how the Brother printer maintains itself. Now 7 years old, and still going strong.

As for cheap ... one HP inkjet cartridge (black ink only; order colour separately) is about $25.

For the Brother printer, 4 black cartridges, two cyan, two magenta and two yellow cartridges (individually wrapped) is $6.25 on Amazon for the ten cartridges.

Yeah, I think I'll stick with the inkjet for a bit longer.

about 9 months ago
top

Ask Slashdot: Life After N900?

KWTm the important thing is scriptability and control (303 comments)

I think it's important to establish what makes the N900 great. Can't speak for the OP, but this is what I'm hoping for in a phone once my N900 finally gives up the ghost. "Hoping," notice I said, but I'm not holding my breath.

1. Scriptability. First and foremost. *I* want to be the one in control of the phone, not some app developer vetted by The Place That Decides What You Can Do With Our --I Mean Your-- Phone (or "AppStore" for short). I want to write a bash script, or a python script, and tell me when my beloved has sent me a SMS containing the word "URGENT".

2. Freedom. Yes, I mean openness as in open source. Yes, I do know not everything in the N900 was open-sourced, but a heck of a lot of it was. That let a lot of people hack it, for the benefit of the community. And it didn't void the warranty. There's something to be said for a phone that does not need you to join the Apple club with a credit card, or sign up with Big Brother Google before using the phone -- you really are independent.

3. Portability of software. It's awesome that I can run Gnumeric on this thing, but even more important that I run Vim.

4. Three things you can change: the cell phone provider, the battery, and the memory storage card. Mainly a criticism compared to the early iPhones; not sure if they still apply. I understand that there are unlocked iPhones now (which still cost more than the N900 did) but you can't change the battery. Android phones will take microSD now, I think?

In fact, to lower my chances of being forced to make do without a good alternative, I bought a second N900, and regularly synchronize the spare so I can have it up and running in case it's needed quickly. I wasn't seeing anything on the horizon, and figured I'd probably have to hang on to my pair of N900's for at least another 3 years. This Slashdot discussion is very useful.

There are, of course, lots to hate about the N900. Most of it deals with the slow swapping caused by the relatively small RAM, versus the large RAM that would be needed by a truly multitasking computer/smartphone. (Compare this with the iPhone that was out at the time, which did not multitask. Do iPhones multitask yet?) The user interface is also unintuitive and poorly thought out. Wish it had been given a chance, but once Elop came on board, there was zero chance of that.

As I've said before, the N900 is a piece of crap --but it's the BEST piece of crap in the world!

about 9 months ago
top

Why Standard Deviation Should Be Retired From Scientific Use

KWTm agree: as bad as "Anti-Fragile" (312 comments)

So you want to retire a statistical term......because people use it incorrectly in economics? Get bent. The standard deviation is a useful tool for statistical analysis of large populations.

Agreed that this is a ridiculous proposal. He probably just wants more publicity.

This was the guy who wrote the book "Anti-Fragile", which I had hoped would educate and broaden my way of thinking, in the same way that the Malcolm Gladwell books ("Tipping Point", "Blink", "Outliers") did. He ended up droning on and on without really making a worthwhile point, and I gave up after a while.

about 9 months ago
top

Ask Slashdot: Best/Newest Hardware Without "Trusted Computing"?

KWTm You still have to show me how to get my keys (290 comments)

I guess I'm trying to cut to the essence of the question: can I get my keys? How can I get my keys?

To clarify: The aspect on which BIOS4breakfast and Alsee disgree is that the former feels that there is not a restriction on obtaining keys as long as they are not obtained from the TPM module, whereas the latter feels that the restriction covers non-TPM aspects as well. Alsee says: "The moment they ... give owners the option to buy chips that come with a printed copy of they keys, then I will [proclaim] that Trusted Computing is wonderful ..." This is the point of contention, and the aspect on which I am focusing.

I guess I should have been more explicit: "Alsee says I can't have the keys to the TPM which comes with the computer I buy, EVEN THROUGH NON-ELECTRONIC MEANS. You disagree with Alsee. We all agree that if I can have the keys, all would be fine."

In the end, it doesn't really matter who agrees with whom where. I want my keys. How do I get them?

about a year ago
top

Ask Slashdot: Best/Newest Hardware Without "Trusted Computing"?

KWTm Prove you're right: Show me how to get my keys (290 comments)

Help me judge which of you is right.

Alsee says I can't have the keys to the TPM which comes with the computer I buy. You disagree with Alsee. We all agree that if I can have the keys, all would be fine.

So, if I buy a computer with TPM, how would I go about getting the keys?

Not a troll. I really want to know, and I'm sure other Slashdotters would thank you, too.

about a year ago
top

Londoners Tracked By Advertising Firm's Trash Cans

KWTm oblig: N900, of course (189 comments)

Just use a N900. It won't unnecessarily broadcast its MAC address.

Of course, then you have to deal with it being so slow and swapping all the time, and the interface that's clunkier than a museum jalopy. As I like to say, the N900 is a piece of crap. But it's the best piece of crap in the world!

about a year ago
top

Ask Slashdot: Should More Math and Equations Be Used In the Popular Press?

KWTm that's rubbish that "mathematicians don't get it" (385 comments)

I doubt most mathematicians really understand the Pythagorean Theorem. You get so used to theories and their application that you fool yourself into thinking you know them. Take manual long division or multiplication for example. We understand how to line up the numbers and perform the operations but prove to me that it works or *why* it works!

I disagree. Some math concepts are deep, but not Pythagoras. Probably the top 5% of high school graduates understand it, and the only reason the majority of the other 95% don't is that they haven't really tried enough. You really can't understand this animated GIF?

You're talking about mathematicians, who have decided that they will be devoted working with math more than any other field, and you think they don't understand? I can't imagine a single mathematician who can't understand Pythagoras.

And long division -- you don't understand why the numbers line up? How it works? I certainly look down on you for not understanding at this moment, but even then I bet if you thought about it for a bit, you'd understand. It's the decimal system -- meaning that the four digits ABCD represent Ax1000 + Bx100 + Cx10 + D -- and the distributive property of multiplication/division over addition/subtraction.

I can't imagine anyone STARTING to learn to become a mathematician without understanding long division (yes, I mean really grasping it, not just how to write the numbers), much less having become a mathematician.

about a year ago
top

Ask Slashdot: What To Do When Another Dev Steals Your Work and Adds Their Name?

KWTm can't you still say "I'm the one who did that"? (480 comments)

Do you know what copyright even means?!?! Clearly you don't. Copyright is how you secure credit for something you created. You boys are DENSE.

As far as I can tell, one major difference between (what I mean by) "credit" and "copyright" is that copyright can be bought or otherwise transacted for money. For example, after you create a work (let's say a book), you can sell the copyright so that someone else (say the publisher) holds the right to receive remuneration for reproducing the work. But that does not take away your ability to say, "You know, I'm the one who did that." See also the comment from the sibling poster amaurea.

If by "credit" you mean "remuneration", then I would agree with your statement that "Copyright is how you secure credit for something you created". But that's not what I'm talking about, nor the OP. Of course there may be circumstances where, due to other contractual obligations, you are not allowed to take credit (undercover ops, ghost writing, etc.), but that's not related to the current situation.

If by "DENSE" you mean "solid; robustly built; able to withstand attacks" then I thank you for the compliment.

about a year ago
top

Ask Slashdot: What To Do When Another Dev Steals Your Work and Adds Their Name?

KWTm agree: this is about credit, not copyright (480 comments)

Agree: this is more about credit than about copyright.

If you had built a bridge for your city, you should be able to list that as one of your accomplishments. It does not mean that you can walk off with the bridge. At the same time, you'd be perfectly justified in getting pissed off if someone else said that it was they, not you, who had built it.

about a year ago
top

Ask Slashdot: Open Source For Bill and Document Management?

KWTm Yes, please post us that script for getting PDFs (187 comments)

If anybody wants some pre-alpha scripts for grabbing their pg&e, comcast, cigna, at&t, schwab, nvenergy statements, let me know.

In a similar vein to http://ask.slashdot.org/comments.pl?sid=3623835&cid=43389299, I would say: yes, please post to pastebin or github or something (maybe even your own Slashdot journal); if you GPL it, someone might even do some fine-tuning for you.

about a year and a half ago
top

Video Editor Kdenlive 0.9.6 Released

KWTm Avidemux can splice/crop without re-encoding (95 comments)

Question, while we're on the subject. I've recently been editing some video, and kdenlive was one of the few video editors I could get to work. However, I've found no way to use the parts of the original video that I haven't modified as they are, without re-encoding. Since most of what I've done is cutting out time ranges from the original footage, using the original data without re-encoding would save a lot of time and quality degradation. Is there any way to do this (using kdenlive or another FOSS video editor)?

Avidemux may be what you want. I use it to crop out parts of video files, by specifying beginning and end points of where I would like video removed. It will then splice it together, and there is a "smart copy" feature where you don't need to re-encode if you choose the same video/audio encoding, and it saves a lot of time. It doesn't always work, though, if there is audio involved: it gets out of sync. If video without audio, it works well, as long as the portions you splice together starts with a key frame (generally where there is a big change from frame-to-frame, such as scene changes, etc.).

Avidemux is a simple video editor, and after splicing/cropping the parts you want, you may want to use a different editor for more advanced stuff.

about a year and a half ago
top

Ask Slashdot: Encrypted Digital Camera/Recording Devices?

KWTm Can we apply this to home security cameras? (285 comments)

This conversation resonates with a topic I've been looking into for some time now: wireless security cameras.

DLink, among others, sells wireless security cameras; they were pretty cheap ($60 before rebate) at Fry's.

Supposedly these are easy to set up: you put one at home, let it hook up to your home wireless router, and it will take pictures which it will upload to DLink; then while you are vacationing in the alps or Bahamas, you can get on the internet and look at how the thieves are (or, more hopefully, are not) breaking into your empty house.

The thing is, not only am I basically telling the Internet world that I have an empty house to break into, but there is a device in my home which could be trying to root my other devices on my network, and which would have a legitimate reason to be talking to some outside agency. For all I know, there could be malware on the camera under the control of DLink, or some renegade (former?) employee at DLink, or not at all related to DLink (the way some iPods came preinstalled with Windows malware).

Is there some sort of encryption and security that can be put into/around these cameras to keep it from doing anything underhanded? The only thing I can think of is to stop it from phoning home altogether (ie. don't use the DLink SeeYourOwnHome.Dlink.com type video upload service and just store stuff on my home server), but maybe other Slashdotters can come up with something more creative.

I admit this is not exactly the type of "Encrypted Digital Camera/Recording Devices" that the OP was talking about (the original question is more about protecting the camera from the outside), but I thought I'd use the opportunity to draw on the Slashdot wisdom about protecting the rest of my home from the camera.

Thanks for any ideas or links you can provide.

about a year and a half ago
top

Book Review: A Practical Guide To Linux Commands, Editors, and Shell Programming

KWTm gcal does sideways format with actual flags (81 comments)

not sure about OS X, but gcal (from GNU) lets you add --type=special (or the "-i" flag). I'm pretty sure you could implement what you find with a shell script looking at $0. So, seems handy, but just a curiosity, I'd say.

about a year and a half ago
top

iPhone Finally Coming To T-Mobile In 2013

KWTm 3G signals for iPhone 4S vs N900 recently improved (154 comments)

A close friend, who uses Tmobile in the SF Bay Area for the Nokia N900, got an iPhone 4S half a year ago. Both phones were bought at the non-subsidized expensive price, unlocked. Where the N900 had been getting 3G signals, in the same location the iPhone 4S would get EDGE only. Siri would be useless. Apparently the iPhone 3G was not on the same frequency as the N900 3G.

That changed about a month ago, where suddenly during the long daily commute up the peninsula (between Silicon Valley and San Francisco), suddenly there would be big areas with the iPhone 4S lighting up with a nice strong 3G signal where there previously had not been any before. We speculated that this was due to MetroPCS merging into Tmobile, but we really didn't know that much (does MetroPCS even have iPhone-compatible 3G?).

about 2 years ago

Submissions

top

Medical "Lifeline" auto-alert device: what

KWTm KWTm writes  |  about 6 years ago

KWTm writes "We had a recent death in the family where an elderly person living alone had fallen and could not get up; by the time she was found days later, it was already too late. This and a number of other similar near misses in our friends and family have prompted us to look into the "Lifeline" type medical alert devices. So far we've set up one arrangement where a signaling device is constantly worn as a pendant around the neck; if the wearer falls and cannot get up, s/he presses the button which signals the staff of the monitoring service to phone and make sure everything is okay. If no one answers the phone in a set amount of tries, they then alert a pre-arranged family member or friend.

This has a number of disadvantages which I figure shouldn't be that hard to overcome.
  1. I'd like a device that can automatically alert even if the wearer becomes unconscious and cannot push the button. It might detect horizontal body position to detect falls, or have motion sensors, etc. (Beeping for 10 minutes first to rule out false alarms, time delay of 8 hours for sleeping, etc.)
  2. It would be nice if it were not tied to any one subscription service. In addition to avoiding vendor lock-in, it would be nice not to worry about the elderly wearer forgetting to pay the subscription fees. Also, one wary family medmber said, "I'm not going to sign up because that would be like telling someone, 'I'm a vulnerable elderly person --come rob me!'"

I figure someone must sell or have put together a system that does not need subscription. Perhaps some device hooked up to the phone, which if activated (and not disarmed within 10 minutes) would phone a pre-set number playing a recording that says, "Emergency services required at (insert address here). This is an automated recording. This is not a joke. This is not a test." Anyone know about this? Anyone tried making one? Anyone hack a Wiimote to trigger a Cheap Old Computer server with a voicemodem running Linux to dial these preset numbers with recordings? Any other ideas are welcome. (Keep in mind that these are generally for elderly people already resistant to the idea that they actually need this sort of device in the first place --makes them feel old and dependent-- so it must be utterly easy to use once set up.)"

top

Mainstream newspaper promotes Linux over Windows

KWTm KWTm writes  |  more than 7 years ago

KWTm (808824) writes "The Independent (UK) newsletter has put out an article comparing Linux to Windows, with a surprisingly positive view for a non-tech-focused publication reviewing desktop Linux. It says about Linux: "It's faster than Windows, it fights viruses — and it's free." In case you were wondering about Whether Linux Is Ready For The Desktop, the author, Jimmy Lee Shreeve, notes "Linux Ubuntu comes with everything you need. There's a powerful office program with desktop publishing, an e-mail/organiser, a web browser and instant messager. You can use your iPod with it, too." Another example of the changing public perception of Linux on the desktop."

Journals

top

GPG signature: first remove whitespace+non-printables

KWTm KWTm writes  |  more than 3 years ago

I've settled on a hopefully credible way to sign my posts without needing to worry about how word-wrapping will mess up the GPG signature. Before signing my posts, I will remove spaces, tabs, newlines, and any other non-printable characters before signing.

That means that the GPG signature of my posts will be invalid as-is (since the text of the posting itself retains the whitespace and non-printables). That means you will have to remove the whitespace/non-printables yourself before running through the GPG verification to see that it is a valid signature. This is how to do it:

put the plain text into PlainTextFile, and the signature (including beginning and end lines) into SignatureFile, and use this command:
tr -cd [:graph:] <PlainTextFile | gpg --verify SignatureFile -

Don't forget the single hyphen at the end of the line, which has a space before it.

top

New OpenPGP key as of 2010-01-15

KWTm KWTm writes  |  more than 4 years ago

Okay, this time I'm remembering to post my new key before my old key expires. So, this replaces the key that I posted in 2008.

The idea is that, if I get kicked out of this account (or someone takes over this account), then I can set up a new Slashdot account and use this key to prove that I'm the same user.

Here it is. In case Slashdot formats it, remember that if there are any extra-long lines, those should be broken up into individual lines. Each line of this OpenPGP key is the same length and contains no spaces.

-----BEGIN PGP PUBLIC KEY BLOCK-----
Version: GnuPG v1.4.6 (GNU/Linux)

mQGiBEtQZNgRBADGdWKqfED1ZHpS+FbcTF5yq0IpaK6YFrUOxtvY6mhNqBFG/+9M
NyZQxQK9Bfx38KBkeu8H4AHeDHnx19wfrRidS2O/6yrcwahN9i/ZAxF2VgjZafZe
PuYnBXCnIkiPcwzBXOo5MkwKHdqKv6yQYbMWBQolVZLJaxG3W7MAGrHF3wCgpXPu /Vm080e3HWIo46C4GyFe9jkEALPNDenlhnQzTC72nhgEbGdVaYZK5k11gw7kaCSZ
YQougUbBzHZX6yrdfvP7qgtudtpUGNgr4CQc56A+2iTn1eibyw/q/FjlW0IcVXez
f3aIxRMbQO+5vXXnDbED5/dZQvSPNuXzR5q25arRYbaMcoH0YzJSQLg/vkbSHwmn
pKkNA/wOBZl3hb23x56WqPaXb0xyHnY6u3/EPd9Lpn54A0vBcvSqzeXIgrESv6kz
bE7C2FMkp+hLP91brPi51gioSrFs6dDrnRw+adUBMjbApjyCy8XN3wk2IV2Oijud
ca9cFo7Ijsp7oXA4B1S8rvtjc/t1uqiC1VIUtiJJ6cCzQQ6ry7Q2S2EgV2FpIFRh
bSAoMjAxMCBKYW4gMTUgS21haWwpIDxrYXdhaS1rYkB0YW1seWxpbi5uZXQ+iGYE
ExECACYFAktQZNgCGyMFCQWjmoAGCwkIBwMCBBUCCAMEFgIDAQIeAQIXgAAKCRAu
dz05U7/JnhBXAJ4namsGw21iZtLo0TsoYlcnmHRyWwCfZuV4cnJFDus6SjVQ0I1S
4ykXq765Ag0ES1Bk5BAIAPrxJ0ljf/g6cmGtqclwe1UBRHusFapl7wjR4TVoysaU
MB7wD8m41gSSKL4LlirlFkL+EceKr7AVxev2af5qgjV+2Px4TS5VvWcFvd1jPcMs
G8D0j9MNM+lE5SSfOwl/gqcaQoMdTsgx21zsDlgJyithiZbWG0mc9UCufdM8+MeE
BpFiykd9rUcuMSMUaYI5dHSZSv+3ZOqpYC1YI4SRYDck/t13GpcA+hElxiJ3PwWv
0PIfb9QkT3qwICgAJ+E28qjrLHqdGy2bALp/HSlLf2QbwUUj6d6DH7lRXzuV0ZqQ
MI7bnouag0b+bIEWUfK7muyEY1P7Nh5WDLmQ1Ta3mDsAAwYH/j1uwBUjpAmj0SiY
pIwVl+nN0fCyVU45gm7CasBENAJ7jG+ZalK83etZ87jhd/T/pIuKvOso1V1g3zAv
O7BVfXbm94ezE9t2tuv/k9QQoMqfajflBv8rHLzPdyzhc5RaAGbV3fP0+7Q5q01W
lpjs4Dkiau/bdg1B7XBdmb8rovB0gfvD+EhWRr+VWWAAV1NdqDKHyYXcdGfRhkjK
lxUDMc7H2l7H1VUoHXnoL1C+8/KOk9RfuKXdIsGztkJrookykOvZ8GTB91eUL2QH
yEfMetNmAG7NzJM2d0NtZyyULRqujPrUcDsJDhD2u66lDH2FElIlT+Zely5FW+aj
WsA9rMiITwQYEQIADwUCS1Bk5AIbDAUJBaOagAAKCRAudz05U7/JnoR8AKCk1m6r
InrF5Gr0XmiJvfi4hAylNwCdGRRJOtJSEogYds0dwvhrrBUv9NA=
=Kmjs
-----END PGP PUBLIC KEY BLOCK-----

top

processing GnuCash files (XML): a Python script

KWTm KWTm writes  |  more than 5 years ago

I was asked for the Python code I wrote to process GnuCash files. I figured I may as well post it publicly so that more than one person can benefit from the work I put in. I cleaned it up a bit, but if you have any questions, please contact me. My program here is called GnuCash_Fix.

As mentioned in the comments: When GnuCash imports QFX/OFX/QIF files (say, for a bank account or credit card account), each transaction is between 2 accounts but GnuCash can only identify 1 of the accounts, that of the bank account/credit card account in question. If the QFX/OFX/QIF file is imported without further intervention, the other account is marked as "Unbalanced" and is assigned to this account (for example, "Unbalanced-USD" if in US Dollars) after doing Main Menu > Actions > Check & Repair > All transactions.

This program tries to identify the other account for each transaction (e.g. "groceries" or "salary") depending on the description of the transaction and optionally the range of the amount. It also finds two complementary unbalanced transactions and matches them together. E.g.
# transaction #1 on 2003-06-21, for $123.45 from Account A to ...somewhere unknown!!!; and then
# transaction #2 on 2003-06-21, for $123.45 from somewhere unknown ... to Account B.
Gee, I wonder what could have happened with those two seemingly unrelated transactions???? (duhhh...)

Works for GnuCash v2.2.6, and also for earlier versions that I used (I can't tell which versions now, but at least one of them was a v1.x.x version).

The main tricky thing I found out from working with GnuCash files is that the tags contain the colon character (':') which Python's XML libraries don't like, so I used "sed" to convert them to double underscores ("__")

In addition to depending on some publicly available software such as "sed" and various Python libraries, it also depends on two Python modules of my own creation. I will post these in subsequent entries. It's easy to rewrite the code so it does not have these dependencies, but if you wanted to make sure the code was working as-is, you'll need these modules. One module is just a debugging script (decide whether to display debugging messages depending on a pre-set "debugging level of detail), and the other is just a personalized version of "getopt". (In fact, I think you can just replace it with the official Python "getopt" --but the version with GNU compatibility.)

Oh, it also depends on a pre-defined file containing parameters that allow this GnuCash_Fix program to identify and classify the transactions. This is called descr_acct.py, and I will post it in a subsequent entry.

#!/usr/bin/env python
#
# Copyright KWTm 2006-2009
 
# released under GPL v3. See http://www.gnu.org/copyleft/gpl.html
# Those who don't like this license may contact me for other licensing options, such as my special I-Don't-Like-GPL-And-Want-To-Give-You-Lots-Of-Money license.
 
#
# When GnuCash imports QFX/OFX/QIF files (say, for a bank account or credit card account), each transaction is between 2 accounts but GnuCash can only identify 1 of the accounts, that of the bank account/credit card account in question. If the QFX/OFX/QIF file is imported without further intervention, the other account is marked as "Unbalanced" and is assigned to this account (for example, "Unbalanced-USD" if in US Dollars) after doing Main Menu > Actions > Check & Repair > All transactions.
 
# This program tries to identify the other account for each transaction (e.g. "groceries" or "salary") depending on the description of the transaction and optionally the range of the amount.
#
# Also finds two complementary unbalanced transactions and matches them together. E.g.
# transaction #1 on 2003-06-21, for $123.45 from Account A to ...somewhere unknown!!!; and then
# transaction #2 on 2003-06-21, for $123.45 from somewhere unknown ... to Account B.
# Gee, I wonder what could have happened with those two seemingly unrelated transactions???? (duhhh...)
 
# Works for GnuCash v2.2.6, and also for earlier versions that I used (I can't tell which versions now, but at least one of them was a v1.x.x version).
 
# Dependencies:
# standard Python libraries, especially xml.dom.ext and xml.dom.minidom
# my own modules: kwdebug, kwgetopt (but this script can be easily rewritten to eliminate these dependences)
# standard GNU/Linux command-line utilities: gzip, sed
 
# The Python xml.dom.minidom module doesn't like colons as part of the XML tag name, so we need to s/:/__/ or something.
 
"""
I'll set up a naming convention:
 
_Ac = instance of Account class
_Class = a class
_Cr = instance of Criterion class
_Gn = instance of Gnucash class
_Tr = instance of Transaction class
_Ts = instance of Timestamp class
_Xn = xml_node
 
"""
 
import solitime
import sys
sys.path.append('/usr/lib/python%s/site-packages/oldxml' % sys.version[:3])
# The above 2 lines are necessary because Ubuntu 8.04 moved the python-xml package, breaking old code. Stupid.
import xml.dom.minidom
import xml.dom.ext
 
import kwdebug
MyDebug = kwdebug.Debug(0)
 
class Global_values_Class:
 
    import kwgetopt
    cmd_line_opts = kwgetopt.kwgetopt("a:d:i:o:")
    # Remember: the colon after the option letters mean that there is an argument after those options
    # -a = account containing unbalanced transactions
    # -d = description / account dictionary file
    # -i = input file
    # -o = output file
    #imbal_acct_re = cmd_line_opts.getopt("-a", r"(?i)imbalance.usd") # The other one is "Unspecified" --make sure it doesn't accidentally refer to "Cu Cdn unspecified", by capitalizing "Unspecified")
 
    descr_acct_fname = cmd_line_opts.getopt("-d", "descr_acct.py")
    input_fname = cmd_line_opts.getopt("-i", "gnucash.xml")
    output_fname = cmd_line_opts.getopt("-o", "gnucash-out.xml")
 
    input_unzipped_fname = input_fname + "_unzipped"
    input_unzipped_nocolons_fname = input_unzipped_fname + "_nocolons"
    output_unzipped_fname = output_fname + "_unzipped"
    output_unzipped_nocolons_fname = output_unzipped_fname + "_nocolons"
 
    preamble=''' GnuCashFIX (with transaction description notes, pair matching and RegExp lookup; updated 954JHq)
: This matches corresponding transactions in Gnucash from a designated account.
: It then takes all the transactions in that account
: and moves them to the appropriate accounts based on the description of
: the transaction.
: (-i) GnuCash input: %s
: (-o) GnuCash output: %s
: (-d) Description -> Account matchfile: %s
: (-a) Account to be fixed will be 'Imbalance-USD'
''' % (input_fname, output_fname, descr_acct_fname )
 
    def __init__(self):
        import os.path
        print(self.preamble)
 
        if os.path.abspath(self.input_fname) == os.path.abspath(self.output_fname) :
            print("\n! *Warning*: output will overwrite the input file %s" % self.input_fname)
        else :
            print(": To force output file to overwrite the input file, use the command-line option '-o %s'" % self.input_fname)
 
        try:
            raw_input("> ENTER to continue, or Ctrl-C to abort")
        except KeyboardInterrupt :
            print("\n! Interrupted by keyboard")
            raise SystemExit
 
Global = Global_values_Class()
 
class GnuCashXML_Class:
 
    # I'll make it so that only this class handles XML, and any other classes do not need to access the XML document at all
 
    acct_denominator_tagname = "act__commodity-scu"
    acct_description_tagname = "act__description"
    acct_id_tagname = "act__id"
    acct_name_tagname = "act__name" # Note that it's abbreviated "act", not "acct", in the GnuCash file
    gnc_acct_tagname = "gnc__account"
    #imbal_acct_re = r"(?i)unspecified"
    #imbal_acct_re = r"Uchq" # for testing only
    split_acct_tagname = "split__account"
    #split_id_tagname = "split__id" # This is not used and may be confused with split_acct_tagname
    split_quantity_tagname = "split__quantity"
    split_value_tagname = "split__value"
    transaction_tagname = "gnc__transaction"
    trn_dateposted_tagname = "trn__date-posted"
    trn_description_tagname = "trn__description"
    trn_id_tagname = "trn__id"
    ts_date_tagname = "ts__date"
 
    # To experiment with XML document in Python IDE, use xml.dom.minidom.parse("/home/kwtm1/finances/gnucash-in.conv")
 
    def __init__(self, in_fname = None):
 
        if None != in_fname:
            MyDebug.debug_msg(": Now converting colons")
            conv_fname = Global.input_unzipped_nocolons_fname
            self.colon_to_double_underscore( in_fname, conv_fname )
            MyDebug.debug_msg(": Now importing %s" % conv_fname )
            self.xmldoc = xml.dom.minidom.parse( conv_fname )
        else:
            self.xmldoc = xml.dom.minidom.Document()
 
        self.acct_dict_AcDict = AccountDict_Class({})
 
    def colon_to_double_underscore(self, in_fname, out_fname):
        conv_cmd = "sed -e 's/:/__/g' %s >%s"
        import os
        os.system( conv_cmd % (in_fname, out_fname) )
 
    def double_underscore_to_colon(self, in_fname, out_fname):
        conv_cmd = "sed -e 's/__/:/g' %s >%s"
        import os
        os.system( conv_cmd % (in_fname, out_fname) )
 
    def convert_acct_xml(self, ac):
        # See convert_trans_xml to see how ridiculously nested the XML is.
        ac_name_contentsnode_val = ac.getElementsByTagName(self.acct_name_tagname)[0].childNodes[0].nodeValue
        ac_id_contentsnode_val = ac.getElementsByTagName(self.acct_id_tagname)[0].childNodes[0].nodeValue
        return Account_Class( ac_id_contentsnode_val, ac_name_contentsnode_val, ac)
 
    def convert_trans_xml(self, tr_Tr):
        # Convert from the transactions themselves back to XML
 
        tr_Tr.node.getElementsByTagName(self.trn_dateposted_tagname)[0].getElementsByTagName(self.ts_date_tagname)[0].childNodes[0].nodeValue = tr_Tr.date
 
        tr_Tr.node.getElementsByTagName(self.trn_id_tagname)[0].childNodes[0].nodeValue = tr_Tr.trn_id
 
        try:
            tr_Tr.node.getElementsByTagName(self.trn_description_tagname)[0].childNodes[0].nodeValue = tr_Tr.description
        except IndexError:
            MyDebug.debug_msg( "Bloody transaction %s had no description so we can't modify that." % tr_Tr ,2)
 
        tr_acctids_list = tr_Tr.node.getElementsByTagName(self.split_acct_tagname) # NOT the self.split_id_tagname which is the id of the split itself, not the accounts involved in the transaction
        tr_values_list = tr_Tr.node.getElementsByTagName(self.split_value_tagname)
        tr_quantities_list = tr_Tr.node.getElementsByTagName(self.split_quantity_tagname) # In all GnuCash transactions I've seen, value = quantity, so we can get the value from either one; but when we modify, then we must update both elements
        tr_num_accts_between = len(tr_acctids_list)
 
        if ( tr_Tr.num_accts_between != tr_num_accts_between ):
            MyDebug.debug_msg( "Funny... I thought transaction %s was between exactly %s accounts, but the XML says it's between %s accounts." % (tr_Tr, tr_Tr.num_accts, tr_num_accts_between ),1)
        else:
 
            if ( tr_num_accts_between >= 1):
                tr_acctids_list[0].childNodes[0].nodeValue = tr_Tr.acct1_Ac.acctid
                tr_values_list[0].childNodes[0].nodeValue = self.derive_valuetext(tr_Tr.value)
                tr_quantities_list[0].childNodes[0].nodeValue= self.derive_valuetext(tr_Tr.value)
 
            if ( tr_num_accts_between >= 2):
                tr_acctids_list[1].childNodes[0].nodeValue = tr_Tr.acct2_Ac.acctid
                tr_values_list[1].childNodes[0].nodeValue = self.derive_valuetext( -tr_Tr.value) # The second value is negative to balance the first.
                tr_quantities_list[1].childNodes[0].nodeValue= self.derive_valuetext( -tr_Tr.value) # The second value is negative to balance the first.
 
    def convert_xml_trans(self, tr):
 
        """
        # The XML is nested to a ridiculous degree. To get the date-posted of the transaction, we would have to do:
        tr_dateposted_list = tr.getElementsByTagName(self.trn_dateposted_tagname)
        tr_dateposted = tr_dateposted_list[0]
        tr_dateposted_date_list = tr_dateposted.getElementsByTagName(self.ts_date_tagname)
        tr_dateposted_date = tr_dateposted_date_list[0]
        tr_dateposted_date_contentsnode_list = tr_dateposted_date.childNodes
        tr_dateposted_date_contentsnode = tr_dateposted_date_contentsnode_list[0]
        tr_dateposted_date_contentsnode_val = tr_dateposted_date_contentsnode.nodeValue
        """
 
        tr_dateposted_date_contentsnode_val = tr.getElementsByTagName(self.trn_dateposted_tagname)[0].getElementsByTagName(self.ts_date_tagname)[0].childNodes[0].nodeValue
        #dateposted = self.extract_date(tr_dateposted_date_contentsnode_val)
        #MyDebug.debug_msg( "Extracted date %s" % dateposted,4)
        # No. There is no need to interpret the date ... yet. Just store the date string as is. When we have to compare, only then do we need to interpret the date string stored in a Transaction.
 
        tr_id_contentsnode_val = tr.getElementsByTagName(self.trn_id_tagname)[0].childNodes[0].nodeValue
 
        try:
            tr_description_contentsnode_val = tr.getElementsByTagName(self.trn_description_tagname)[0].childNodes[0].nodeValue
        except:
            tr_description_contentsnode_val = ""
            MyDebug.debug_msg( "Bloody transaction %s has no description!?" % tr_id_contentsnode_val ,2)
 
        tr_acctids_list = tr.getElementsByTagName(self.split_acct_tagname) # NOT the self.split_id_tagname which is the id of the split itself, not the accounts involved in the transaction
        tr_num_accts_between = len(tr_acctids_list)
 
        if ( 2 != tr_num_accts_between ):
            MyDebug.debug_msg( "Funny... transaction %s (%s) is not between exactly 2 accounts, but with %s accounts." % (tr_id_contentsnode_val, tr_description_contentsnode_val, tr_num_accts_between ),4)
 
        if ( tr_num_accts_between >= 1):
            tr_acctid1_contentsnode_val = tr_acctids_list[0].childNodes[0].nodeValue
            acct1_Ac = self.acct_dict_AcDict.get(tr_acctid1_contentsnode_val, None) # Get the account, or return None if no such account
        else:
            acct1_Ac = None
 
        if ( tr_num_accts_between >= 2):
            tr_acctid2_contentsnode_val = tr_acctids_list[1].childNodes[0].nodeValue
            acct2_Ac = self.acct_dict_AcDict.get(tr_acctid2_contentsnode_val, None)
        else:
            acct2_Ac = None
 
        tr_splitvalue_contentsnode_val = tr.getElementsByTagName(self.split_value_tagname)[0].childNodes[0].nodeValue
        value = self.extract_value(tr_splitvalue_contentsnode_val)
 
        # Now to create an instance of Transaction_Class based on the values above
        return Transaction_Class(tr, acct1_Ac, acct2_Ac, tr_dateposted_date_contentsnode_val, tr_description_contentsnode_val, tr_num_accts_between, tr_id_contentsnode_val, value)
 
    def derive_valuetext(self, value, denominator=100):
        return "%d/%d" % (value*denominator, denominator)
 
    def extract_value(self, text):
        # The value text is something like "12345/100" which means "$123.45", but we have to divide numerator by denominator.
 
        try:
            return eval( text.strip() + ".0")
            # If we don't add ".0", then we will be doing integer arithmetic
        except:
            # Okay, just evaluating didn't work, so now we actually have to decode the thing
 
            import re
            value_re = "(\D|^)(?P<numerator>\d+)/(?P<denominator>\d+)(\D|$)"
            match_result = re.search( value_re, text)
            MyDebug.debug_msg("Extracting value from %s" % text,5)
            if match_result == None:
                return None
 
            numerator = float( match_result.groupdict("0").get( "numerator" ) )
            # Get the text that matches the groupname "numerator" in the regular expression (if can't find it, default to "0") and convert it to a float(ing point number) in preparation for calculating the value of this transaction.
            denominator = float( match_result.groupdict("100").get( "denominator" ) )
 
            MyDebug.debug_msg("Value is %s over %s" % (numerator,denominator),5)
            return (numerator/denominator)
 
    def get_acct_dict(self):
        MyDebug.debug_msg("Creating dict of all accounts", 1)
        acct_dict = AccountDict_Class({})
        acct_list = self.xmldoc.getElementsByTagName(self.gnc_acct_tagname)
        for acct in acct_list:
            acct_dict.add_account( self.convert_acct_xml(acct) )
 
        self.acct_dict_AcDict = acct_dict
 
    def get_all_trans_list(self):
        MyDebug.debug_msg("Creating list of all transactions", 1)
        trn_list = self.xmldoc.getElementsByTagName(self.transaction_tagname)
        all_trans_list = Trans_list_Class([])
        for trn in trn_list:
            all_trans_list.append( self.convert_xml_trans(trn) )
        return all_trans_list
 
    def get_unbal_trans_list(self):
        raise NotImplementedError, "Hey, this function is not going to be used."
        #stub
        return []
 
    def modify_original_trans(self, trans_list_TrList):
        for trans_Tr in trans_list_TrList:
            if trans_Tr.need_to_delete_original():
                MyDebug.debug_msg("I would delete %s." % (trans_Tr) ,4)
                trans_Tr.node.parentNode.removeChild(trans_Tr.node)
                trans_Tr.node.unlink()
            elif trans_Tr.need_to_update_original():
                MyDebug.debug_msg("I would update %s." % (trans_Tr) ,4)
                self.convert_trans_xml(trans_Tr)
            else:
                MyDebug.debug_msg("No changes in %s." % (trans_Tr) ,4)
        #stub
 
    def save_changes(self):
        #stub
        pass
 
    def write_to_file(self, out_fname):
        conv_fname = Global.output_unzipped_nocolons_fname
        MyDebug.debug_msg("Writing to file %s" % conv_fname ,1)
        xml.dom.ext.PrettyPrint(self.xmldoc, open(conv_fname, "wb"))
        MyDebug.debug_msg("Converting to %s" % out_fname ,1)
        self.double_underscore_to_colon(conv_fname, out_fname)
 
class Account_Class():
 
    acctid = None
    acctname = None
    acctnode = None
 
    def __init__(self, acctid=None, acctname=None, acctnode=None):
        self.acctid = acctid
        self.acctname = acctname
        self.acctnode = acctnode
 
    def __eq__(self, other):
        if type(other) == type(None):
            return False
 
        if type(other) != type(self):
            raise TypeError, "%s needs to be of type %s" % (other, type(self))
 
        return self.acctid == other.acctid
 
    def __ne__(self, other):
        return not (self == other)
 
    def __repr__(self):
        return "Account %s, which is named %s and resides in node %s" % (self.acctid, self.acctname, self.acctnode)
 
class AccountDict_Class(dict):
    # The dictionary key shall be the account id; the value corresponding to that key shall be an instance of Account_Class that has info about that account.
 
    def __init__(self, *other_args):
        dict.__init__(self, *other_args)
 
    def add_account(self, account):
        self[ account.acctid ] = account
 
    def lookup_by_re(self, name_re):
        # Yes, partial matches do match (ie. if name_re is "bcd" it will find account "abcde")
        import re
        for acct in self.values():
            match_result = re.search( name_re, acct.acctname )
            if match_result != None:
                break
 
        if match_result == None:
            # We didn't find it. We got here because the loop ended, not because we found it and broke out of the loop
            acct=None
 
        return acct
 
class Criteria_Class:
 
    def __init__(self, criteria):
        import re
 
        self.re_c = None
        self.lower_lim = None
        self.upper_lim = None
 
        re_text = "^$" # Initialize to a regular expression that represents the null string
        if str == type(criteria):
            re_text = criteria
        elif tuple == type(criteria) or list == type(criteria):
            if 3 <= len(criteria):
                (re_text, self.lower_lim, self.upper_lim) = tuple(criteria)[:3]
            elif 2 == len(criteria):
                (re_text, self.lower_lim) = tuple(criteria)[:2]
                self.upper_lim = self.lower_lim
            elif 1 == len(criteria):
                re_text = criteria[0]
            # If 0 == len(criteria_text), then we can just use the default values, so we don't have to do anything.
 
        self.re_c = re.compile(re_text)
 
    def __repr__(self):
        return "<criteria: re_c %s, lower_lim %s, upper_lim %s>" % (self.re_c, self.lower_lim, self.upper_lim)
 
    def fulfills_limit_criteria(self, trn_Tr):
 
        # Based on the description of the transaction, such as "MEGA FUEL",
        # as well as upper/lower limits of the transaction amount,
        # figure out whether a given transaction fulfills the criteria
 
        # The order of these tests matter as we go through trying to decide whether the transaction fulfills criteria
 
        #MyDebug.debug_msg("crit.re_c is %s" % self.re_c,3)
        # First check that there's a valid description in the criterion
        if None == self.re_c:
            return False
 
        # Now see if the criterion description matches that of the transaction
 
        if None == self.re_c.search(trn_Tr.description):
            MyDebug.msg_no_cr("N",4)
            return False
        MyDebug.debug_msg("crit matches %s" % trn_Tr.description,4)
        # Now see if there's a valid upper limit (if there's just a lower limit, the upper limit should already have been set to equal the lower limit)
        if None == self.upper_lim:
            MyDebug.msg_no_cr("-",4)
            return True
        elif trn_Tr.value > self.upper_lim:
            MyDebug.msg_no_cr(">",4)
            return False
        elif trn_Tr.value < self.lower_lim:
            MyDebug.msg_no_cr("<",4)
            return False
        else:
            MyDebug.msg_no_cr("=",4)
            return True
 
class Criteria_list_Class(list):
    # contains these properties:
    # crit_dict = the compiled version of the critera. Key = compiled regexp; Value = the account
    # text_dict = the text of the criteria, in a dictionary, read from file
 
    def __init__(self, descr_fname=None, account_dict=None, *other_args):
        list.__init__(self, *other_args)
 
        if descr_fname != None:
            print(": Reading account description from file %s" % descr_fname)
            try:
                self.text_list = self.eval_py_expr_file(descr_fname)
            except TypeError, errmsg:
                print("\n! ERROR: %s" % errmsg)
                print("! Looks like account description file %s is not a correct Python expression. Most likely there is a comma missing at the end of a line." % Global.descr_acct_fname)
                print("! Try looking for the regexp '^[^#]*\)[^,]*($|#)'")
                print("! (that is, a line where a comma does not occur between the closing parenthesis and the end of line/start of comment)")
                raise SystemExit
            except SyntaxError, errmsg:
                print("\n! ERROR: %s" % errmsg)
                print("! Account description file %s is not a correct Python expression. This is not just a comma missing at the end of a line; some extra character got inserted or something." % Global.descr_acct_fname)
                print("! Try deleting parts of the file and retrying, to narrow down where the problem might be.")
                raise SystemExit
 
        else:
            self.text_list = []
 
        if account_dict != None:
            self.acct_dict = account_dict
        else:
            self.acct_dict = AccountDict_Class({})
 
        MyDebug.debug_msg(": initialized crit list" ,2)
        self.initialize_crit_list()
 
    def eval_py_expr_file(self, fname):
        file = open(fname, 'rb')
        py_expr = file.read()
        return eval(py_expr)
 
    def initialize_crit_list(self):
        import re
 
        try:
            for text_list_element in self.text_list:
                (crit_key, acctname_re) = text_list_element[:2]
                acct_Ac = self.acct_dict.lookup_by_re(acctname_re)
                if acct_Ac == None :
                    print("\n! Unidentified account '%s' --ignoring criteria '%s'" % (acctname_re, crit_key))
                else :
                    criteria_Cr = Criteria_Class(crit_key)
                    self.append( (criteria_Cr, acct_Ac) + tuple(text_list_element[2:]) )
                    # Add the criteria and the account name, but also any other elements in the tuple which might correspond to other features
 
        except ValueError, errmsg:
            print("\n! ERROR: %s" % errmsg)
            print("! Is the account description file %s in the wrong format? As of 2009-05-03, it needs to be a list, not a dictionary." % Global.descr_acct_fname)
            raise SystemExit
 
class Timestamp_Class(solitime.Solitime_class):
 
    def __init__(self, *other_args):
        return solitime.Solitime_class.__init__(self, *other_args)
 
    def formatted_timestamp(self, *other_args):
        return solitime.Solitime_class.formatted_solitime(self, *other_args)
 
class Transaction_Class():
 
    node = None
    acct1_Ac = None
    acct2_Ac = None
    #book_Gn = None
    date = None
    description = None
    num_accts_between = 0
    trn_id = None
    value = None
 
    def __init__(self, node=None, acct1_Ac=None, acct2_Ac=None, date=None, description=None, num_accts_between=0, trn_id=None, value=None):
        self.node = node
        self.acct1_Ac = acct1_Ac
        self.acct2_Ac = acct2_Ac
        self.date = date
        self.description = description
        self.num_accts_between = num_accts_between
        self.trn_id = trn_id
        self.value = value
        self.update_flag = False # If this is None, then the transaction is to be deleted
 
    def __repr__(self):
        return "Transaction, with id %s, for amt$ %s, on date %s, between acct %s and acct %s, covering %s accounts, described as %s. Stored in XML node %s." % (self.trn_id, self.value, self.date, self.acct1_Ac, self.acct2_Ac, self.num_accts_between, self.description, self.node)
 
    def __str__(self):
        return "Transaction %s: on %s, $%s from %s to %s." % (self.trn_id, self.date, self.value, self.acct1_Ac.acctname, self.acct2_Ac.acctname)
 
    def extract_timestamp(self):
        mytimestamp_Ts = Timestamp_Class()
        mytimestamp_Ts.set_time_from_string(self.date)
        MyDebug.debug_msg("Converting date from '%s' to '%s'" % (self.date, mytimestamp_Ts.formatted_timestamp()) ,4)
        return mytimestamp_Ts
 
    def flag_to_delete_original(self):
        self.update_flag = None
 
    def flag_to_preserve_original(self):
        self.update_flag = False
 
    def flag_to_update_original(self):
        self.update_flag = True
 
    def need_to_delete_original(self):
        return ( None == self.update_flag)
 
    def need_to_update_original(self):
        return ( True == self.update_flag)
 
class Trans_list_Class(list):
 
    def __init__(self, *other_args):
        list.__init__(self, *other_args)
 
    def find_matching_trans(self, unbal_trans_Tr):
        #found_it = False # We don't need this yet
        for poss_matching_trans_Tr in self:
            if self.the_dates_match(unbal_trans_Tr, poss_matching_trans_Tr) and \
                self.the_accounts_and_amounts_match(unbal_trans_Tr, poss_matching_trans_Tr):
                return poss_matching_trans_Tr
 
        return None
 
    def the_accounts_and_amounts_match(self, unbal_trans_Tr, poss_matching_trans_Tr):
        # Two cases: both transactions share the same Acct1, or same Acct2; then first amount should be negative of the second amount
        # Otherwise if the Acct1 of one is the same as the Acct2 of the other, or vice versa, then the amounts should be identical (not negatives of each other)
        if (unbal_trans_Tr.acct1_Ac == poss_matching_trans_Tr.acct1_Ac) or (unbal_trans_Tr.acct2_Ac == poss_matching_trans_Tr.acct2_Ac):
            return unbal_trans_Tr.value == -(poss_matching_trans_Tr.value)
        elif (unbal_trans_Tr.acct1_Ac == poss_matching_trans_Tr.acct2_Ac) or (unbal_trans_Tr.acct2_Ac == poss_matching_trans_Tr.acct1_Ac):
            return unbal_trans_Tr.value == poss_matching_trans_Tr.value
 
    def the_dates_match(self, unbal_trans_Tr, poss_matching_trans_Tr):
        unbal_trans_timestamp = unbal_trans_Tr.extract_timestamp()
        poss_matching_trans_timestamp = poss_matching_trans_Tr.extract_timestamp()
 
        return_val= abs(unbal_trans_timestamp.time_tuple[0]-poss_matching_trans_timestamp.time_tuple[0]) <= 1 and \
            unbal_trans_timestamp.time_tuple[1] == poss_matching_trans_timestamp.time_tuple[1] and \
            unbal_trans_timestamp.time_tuple[2] == poss_matching_trans_timestamp.time_tuple[2]
        # The dates will be considered matching if the month and day match exactly, and if the years match or are exactly 1 apart (my own special code)
        return return_val
 
class Transfix_Class():
 
    def __init__(self):
        pass
 
    def classify_trans_by_descr(self, trn_Tr):
        # Based on the description of the transaction, such as "MEGA FUEL",
        # figure out which income/expense account this should be, such as "Expenses:consumables:gasoline"
        #description_node = trn.getElementsByTagName(Global.trn_description_tagname)
        #if 0 >= len(description_node):
        # raise AccountError, "Can't find transaction description"
        #description = description_node[0].childNodes[0].nodeValue
 
        return_acct_Ac = None
        return_trans_descr = None
        for element in self.criteria_list_CrList:
            (criterion_Cr, acct_Ac) = element[:2]
            if criterion_Cr.fulfills_limit_criteria( trn_Tr ):
                # The transaction description matches, so let's see what account this transaction should be under
                return_acct_Ac = acct_Ac
                if len(element) >=3:
                    return_trans_descr = element[2] + " - " + trn_Tr.description
                    # ToDo: change this to a more sophisticated substitution
                break
 
        return (return_acct_Ac, return_trans_descr)
 
    def classify_trans_of_translist_by_descr(self, remaining_unbal_trans_list_TrList):
 
        print("")
        for unbal_trans_Tr in remaining_unbal_trans_list_TrList:
            (corrected_acct_Ac, corrected_trans_descr) = self.classify_trans_by_descr(unbal_trans_Tr)
            if None != corrected_acct_Ac:
                # Do this only if we've actually found the corrected acct id, or else leave the acct id alone
 
                # But is it acct1 or acct2 of the transaction that we need to correct? The following takes care of this.
                if unbal_trans_Tr.acct1_Ac in self.unbal_acct_list:
                    unbal_trans_Tr.acct1_Ac = corrected_acct_Ac
                elif unbal_trans_Tr.acct2_Ac in self.unbal_acct_list:
                    unbal_trans_Tr.acct2_Ac = corrected_acct_Ac
 
                if corrected_trans_descr != None:
                    unbal_trans_Tr.description = corrected_trans_descr
 
                unbal_trans_Tr.flag_to_update_original()
                MyDebug.msg_no_cr("!", 8)
                print(": (%s) $%0.2f\t%s -> '%s'" % (unbal_trans_Tr.date, unbal_trans_Tr.value, unbal_trans_Tr.description, corrected_acct_Ac.acctname))
            else:
                print("! (%s) $%0.2f *\t*\t* no account found for '%s'" % (unbal_trans_Tr.date, unbal_trans_Tr.value, unbal_trans_Tr.description))
 
    def extract_unbal_trans_list(self, all_transactions, unbal_acct_list):
        # First, clean up unbal_acct_list because it might contain None's
        no_more_Nones = False
        while not no_more_Nones:
            try:
                unbal_acct_list.remove(None)
            except:
                # We couldn't remove any more Nones from the list, so we must have gotten rid of all of them
                no_more_Nones = True
 
        MyDebug.debug_msg("After removing Nones, our unbal_acct_list is %s" % unbal_acct_list,4)
 
        unbal_trans_list = Trans_list_Class([])
        for tr in all_transactions:
            MyDebug.msg_no_cr("t", 3)
            #MyDebug.debug_msg("Checking tr %s." % tr,4)
            for unbal_acct in unbal_acct_list:
                MyDebug.debug_msg("Checking with account %s." % unbal_acct,4)
                if (tr.acct1_Ac == unbal_acct) or (tr.acct2_Ac == unbal_acct):
                    MyDebug.debug_msg("Found that transaction %s is unbalanced." % tr, 3)
                    unbal_trans_list.append(tr)
 
        return unbal_trans_list
 
    def find_matching_pairs_of_unbal_trans(self, unbal_trans_list_TrList):
    #def get_book_fn(self):
    # return "gnucash.xml"
        self.matched_unbal_trans_list = [] # This will contain tuples (pairs) of the matched accounts
        self.unmatched_unbal_trans_list_TrList = Trans_list_Class([])
        self.already_matched_unbal_trans_list_TrList = Trans_list_Class([])
        for unbal_trans_Tr in unbal_trans_list_TrList:
            if unbal_trans_Tr not in self.already_matched_unbal_trans_list_TrList :
                match_unbal_trans_Tr = unbal_trans_list_TrList.find_matching_trans(unbal_trans_Tr)
                if None != match_unbal_trans_Tr:
                    # We found a match! Now put the match in already_matched_unbal_trans_list_TrList so we don't try to look for a match (which would produce duplicate match)
                    self.already_matched_unbal_trans_list_TrList.append( match_unbal_trans_Tr )
                    self.matched_unbal_trans_list.append( (unbal_trans_Tr, match_unbal_trans_Tr) )
                    #my_gnucash_book_Gn.reconcile(unbal_trans_Tr, match_unbal_trans_Tr)
                    MyDebug.debug_msg("Found match. Currently %s unbalanced and %s pairs of matched transactions." % ( len(self.unmatched_unbal_trans_list_TrList),len( self.matched_unbal_trans_list ) ) ,4)
                else:
                    self.unmatched_unbal_trans_list_TrList.append( unbal_trans_Tr )
                    MyDebug.debug_msg("Could not find match for %s" % unbal_trans_Tr,4)
                    MyDebug.debug_msg("No match. Currently %s unbalanced and %s pairs of matched transactions." % ( len(self.unmatched_unbal_trans_list_TrList),len( self.matched_unbal_trans_list ) ) ,4)
        print(": Could not find match for %s unbalanced transactions." % len(self.unmatched_unbal_trans_list_TrList))
        print(": %s pairs of transactions to be reconciled." % len( self.matched_unbal_trans_list ))
 
        for (tr1_Tr, tr2_Tr) in self.matched_unbal_trans_list:
            self.reconcile_matching_unbal_trans(tr1_Tr, tr2_Tr)
 
        return self.unmatched_unbal_trans_list_TrList
 
    def reconcile_matching_unbal_trans(self,tr1_Tr, tr2_Tr):
        # The goal is to modify the transaction where the *second* account (not the first) is the shared one.
        # Then we delete the transaction where the first account is the shared one. So, if we have:
        # tr1: $10 from Acct A to Acct B
        # tr2: $10 from Acct B to Acct C
        # Then we delete tr2, and change tr1 to: $10 from Acct A to Acct C
 
        if (tr1_Tr.acct2_Ac == tr2_Tr.acct1_Ac):
            tr1_Tr.acct2_Ac = tr2_Tr.acct2_Ac
            tr2_Tr.acct1_Ac = tr1_Tr.acct1_Ac
        elif (tr1_Tr.acct2_Ac == tr2_Tr.acct2_Ac):
            tr1_Tr.acct2_Ac = tr2_Tr.acct1_Ac
            tr2_Tr.acct2_Ac = tr1_Tr.acct1_Ac
        elif (tr1_Tr.acct1_Ac == tr2_Tr.acct2_Ac):
            tr1_Tr.acct1_Ac = tr2_Tr.acct1_Ac
            tr2_Tr.acct2_Ac = tr1_Tr.acct2_Ac
        elif (tr1_Tr.acct2_Ac == tr2_Tr.acct2_Ac):
            tr1_Tr.acct2_Ac = tr2_Tr.acct1_Ac
            tr2_Tr.acct2_Ac = tr1_Tr.acct1_Ac
        else:
            raise AssertionError, "Looks like these aren't really matching transactions: %s and %s" % (tr1_Tr, tr2_Tr)
 
        # At this point each of the two transactions where one of the accounts was the Imbalance account has had its Imbalance account replaced with the correct account. So there are 2 duplicate transactions and we need to get rid of one.
        # Which do we keep? The one with the earlier date, since my special code is that I will put a date one year ahead. That is, if I deposit a cheque for $123.45 on 2003-06-21, then I'll put a reminder transaction that on 2004-06-21, $123.45 went somewhere. So even if the cheque doesn't show up right away, I will have a reminder that there is a transaction there... somewhere. I think. Does that work? Whatever. Just do it.
        if tr1_Tr.date < tr2_Tr.date :
            tr1_Tr.flag_to_update_original()
            tr2_Tr.flag_to_delete_original()
        else:
            tr2_Tr.flag_to_update_original()
            tr1_Tr.flag_to_delete_original()
 
        MyDebug.debug_msg("Reconciled transactions to: \n%s, and \n%s" % (tr1_Tr, tr2_Tr) ,4)
 
    def xml_manip_main(self):
        import os
 
        print(": Reading from finances file '%s'" % Global.input_fname)
        MyDebug.debug_msg("Now unzipping")
        result = os.system( "gunzip --to-stdout %s >%s" % (Global.input_fname, Global.input_unzipped_fname) )
        MyDebug.debug_msg("unzip result is %s" % result)
        if result != 0 :
            print(": %s does not appear to be zipped. Trying as unzipped file." % Global.input_fname)
            input_unzipped_fname = Global.input_fname
        else :
            input_unzipped_fname = Global.input_unzipped_fname
 
        self.my_gnucash_book_Gn = GnuCashXML_Class(input_unzipped_fname)
        self.my_gnucash_book_Gn.get_acct_dict()
        self.acct_dict_AcDict = self.my_gnucash_book_Gn.acct_dict_AcDict
        #MyDebug.debug_msg("Formed account dictionary of size %s" % len(acct_dict_AcDict) ,1)
        print(": Found %s accounts in %s" % (len(self.acct_dict_AcDict), Global.input_fname) )
        MyDebug.debug_msg(self.acct_dict_AcDict,4)
 
        self.all_trans_list_TrList = self.my_gnucash_book_Gn.get_all_trans_list()
        #MyDebug.debug_msg("There are %s transactions in total." % len(all_trans_list_TrList) ,1)
        print(": Found %s transactions in %s" % (len(self.all_trans_list_TrList) , Global.input_fname) )
 
        self.unbal_acct_list = [self.acct_dict_AcDict.lookup_by_re("Unspecified"),
                    self.acct_dict_AcDict.lookup_by_re("Imbalance-USD")]
        MyDebug.debug_msg("Unbal acct list is %s" % self.unbal_acct_list,2)
 
        self.unbal_trans_list_TrList = self.extract_unbal_trans_list( self.all_trans_list_TrList, self.unbal_acct_list )
        print(": There are %s unbalanced transactions." % len(self.unbal_trans_list_TrList))
 
        # We will store the original unbal_trans_list,
        # and then do various things like match up corresponding transactions and switch the account to the right account based on the description
        # Each time we do this, we maintain a list of transactions that are still not fixed, and pass it to the next function
 
        # Step 1
        #remaining_unmatched_unbal_trans_list_TrList = self.unbal_trans_list_TrList
        remaining_unmatched_unbal_trans_list_TrList = self.find_matching_pairs_of_unbal_trans(self.unbal_trans_list_TrList)
        #MyDebug.debug_msg(": skipping matching pairs. Revert this later on.")
 
        #remaining_unmatched_unbal_trans_list_TrList = self.fix_account_if_needed_based_on_descr(remaining_unbal_trans_list_TrList)
 
        # Step 2
        self.criteria_list_CrList = Criteria_list_Class(Global.descr_acct_fname, self.acct_dict_AcDict)
        remaining_unmatched_unbal_trans_list_TrList = self.classify_trans_of_translist_by_descr(remaining_unmatched_unbal_trans_list_TrList )
 
        self.my_gnucash_book_Gn.modify_original_trans( self.unbal_trans_list_TrList )
 
        print(": Done. Now writing to file '%s'" % Global.output_fname)
        self.my_gnucash_book_Gn.write_to_file(Global.output_unzipped_fname)
        MyDebug.debug_msg("Now zipping")
        result = os.system( "gzip --to-stdout %s >%s" % (Global.output_unzipped_fname, Global.output_fname) )
        MyDebug.debug_msg("zip result is %s" % result)
        if result != 0 :
            print(": %s may not have been zipped properly. Try using unzipped intermediate file %s" % (Global.output_fname, Global.output_unzipped_fname) )
 
def main():
    mytransfix = Transfix_Class()
    mytransfix.xml_manip_main()
 
if __name__ == "__main__":
    main()

End of listing for my Python program.

top

detecting subtle changes in Terms & Conditions

KWTm KWTm writes  |  more than 5 years ago

This is to check for subtle changes in text that you see often, such as Terms & Conditions that pop up every time you use a frequently-used Web service. It will alert you to small changes that you might not otherwise notice because you habitually click on the "I Agree To These Terms And Conditions" button without going through all the text each time. Simply select the text and copy to the clipboard, and then run the script.

This shell script compares what's in the clipboard to text files in a certain directory; in this case it's ~/Documents/terms_and_conditions. This is where I would store the T&C text from various web sites. When it finds a similar match, it does a diff to look for minor changes. If there is no exact match, it offers to store the copied text in a new text file so you can compare the current version to future versions.

It works with the KDE clipboard, Klipper. Those of you who use GNOME or some other clipboard system will need to modify the line that says: "dcop klipper klipper getClipboardContents >$TEMP_FILE". (In fact, I think those of you using KDE 4 will need to modify this line, too.)

#!/bin/sh
 
COMPARISON_DIR="$HOME/Documents/terms_and_conditions"
COMPARISON_LENGTH=160
SAVE_FOR_FUTURE_FN="clipboard_saved"
TEMP_FILE="temp.tmp"
TEMP_FILEHEAD="temp_head.tmp"
TEMP_CANDIDATE_HEAD="temp_cand_head.tmp"
 
QUIET_FLAG=`echo " $@" | grep -cE " -Q"`
IMMEDIATE_FLAG=`echo " $@" | grep -cE " -I"`
 
if [ $QUIET_FLAG -le 0 ] ; then :
    # The Quiet cmd line argument was NOT specified.
    echo " "
    echo " CompareTermsAndConditions [options] [-I] [-Q] (cannot specify '-IQ' together; '-I -Q' is ok. )"
    echo ": This compares what's in the KDE klipboard with files in $COMPARISON_DIR"
    echo ": automatically searching all files to see which file might be similar or identical."
    echo ": This is to look for changes in those Terms & Conditions on web sites which show them repeatedly, to make sure that after you stopped bothering to check for changes, they don't sneak in unnoticeable changes to the Terms & Conditions."
    echo ": Make sure you have copied the text to clipboard."
    echo ": '-Q' will suppress output to stdout; '-I' will skip the prompt and proceed immediately"
    if [ $IMMEDIATE_FLAG -le 0 ] ; then :
        # The immediate line argument was NOT specified.
        read -p "> ENTER to continue, or Ctrl-C to abort" RESULT
        echo "\n"
    fi
fi
 
OLD_DIR=$PWD
cd $COMPARISON_DIR
 
dcop klipper klipper getClipboardContents >$TEMP_FILE
PATTERN=`grep --max-count=1 '[A-Za-z]' <$TEMP_FILE`
echo ": We will look for files that start with '$PATTERN'"
FILELIST=`grep --fixed-strings -- "$PATTERN" * | sed -r -e 's/([^:]*):.*$/\1/' | uniq`
echo ": Will compare your text to the following similar-looking files:"
echo $FILELIST
 
FOUND_IDENTICAL="False"
for FN in $FILELIST; do :
    if [ $TEMP_FILE != $FN -a $TEMP_FILEHEAD != $FN -a $TEMP_CANDIDATE_HEAD != $FN ]; then :
        #echo " "
        head --bytes=$COMPARISON_LENGTH $TEMP_FILE > $TEMP_FILEHEAD
        head --bytes=$COMPARISON_LENGTH $FN > $TEMP_CANDIDATE_HEAD
        diff --ignore-all-space --brief $TEMP_FILEHEAD $TEMP_CANDIDATE_HEAD > /dev/null
        if [ $? -eq 0 ]; then :
            # Yes, $FN is really worth comparing.
            # We check this just to make sure it's not spuriously matching blank lines in the cliptext.
            echo ": Now comparing with $FN"
            diff --report-identical-files --ignore-all-space $TEMP_FILE $FN
            if [ $? -eq 0 ]; then :
                FOUND_IDENTICAL="True"
                echo ": --- IDENTICAL! ---"
                echo " "
            fi
        else :
            echo ": Skipping $FN, which isn't really all that similar."
        fi
    fi
done
 
if [ "False" = $FOUND_IDENTICAL ]; then :
    echo " "
    if [ $IMMEDIATE_FLAG -le 0 ] ; then :
        # Immediate flag was NOT specified, so we can be interactive and ask if wants to save text to file for future comparison
        echo "! NO MATCH. Want to save the current text for future comparison? (Specify '-I' next time to automatically say yes.)"
        read -p "> ('y'=yes, else no):" RESULT
        if [ "a_y" != a_$RESULT ]; then :
            exit # Otherwise proceed to save
        fi
    fi
    TIMESTAMP=`date +%y%m%d%H%M%S`
    mv $TEMP_FILE "$SAVE_FOR_FUTURE_FN-$TIMESTAMP.txt"
    echo ": Now saving to file $PWD/$SAVE_FOR_FUTURE_FN-$TIMESTAMP.txt"
fi

top

KWTm KWTm writes  |  more than 6 years ago

Well, gee, the plan was to sign this new key before the old key lapsed, but lazy me... anyway, at least I'm still using the same user account. The idea is that, if I get kicked out of this account (or someone takes over this account), then I can set up a new Slashdot account and use this key to prove that I'm the same user.

Here it is. In case Slashdot formats it, remember that if there are any extra-long lines, those should be broken up into individual lines. Each line of this OpenPGP key is the same length and contains no spaces.

-----BEGIN PGP PUBLIC KEY BLOCK-----
Version: GnuPG v1.4.6 (GNU/Linux)

mQGiBEgo38YRBADIxPJE2WJjqCnqp7dshy772sOfpdud15I+pJSRVq8/ED8R6/A/
NFe8GCqL5//IRvj2WaJtd8OLVfJtzb+4Q8kTvgtsBCQU02KyBGEBGhyfxwUBxEYP
Ynwu2GdhkodLRKOEdmLuegvirmxbjFr2Bb408lRG7UkUbWH2uZRTvIoRYwCg8Pib
56ZSgLIRZbip/MLnavMtrKUD/RHCrUDJg09zmN9P0DwaFdod1Tp5xNg/ko3uGli9
0+WjPEMCQCK51nwNEnPijhbRq38f/Jbut/fFJlawwEyrEaufIfGr+FwLX2zKhgSM
a57q/b4dzz9f7Ky4vwoJK+Oh8zlISL+uZs5RPRep+GjYUJTjMa8D+R9txb7gX/tX
17coBAC7Akp8SFiXvbylN1eSzvWUYiwoD7R9m4naLhHpIcf1A6J1H1Tra/ZvsMcp
k+Cz6gy3FrZn2Az6Gi2+gvJE04ma0loKN5kN90tNm9BEM24/25Ontef1ubL47Dty
iY8MNfTFzcTCaapmwGJQ4h017p5Z8hyXFglz4Stdpa7CgK1GsLQ2S2EgV2FpIFRh
bSAoMjAwOCBNYXkgMTIgS21haWwpIDxrYXdhaS1rYkB0YW1seWxpbi5uZXQ+iGYE
ExECACYFAkgo38YCGyMFCQPCZwAGCwkIBwMCBBUCCAMEFgIDAQIeAQIXgAAKCRBx /pIpG5px4O6DAJwNbNQEUakeF6mPRb3Q4T18ZAP1YgCg5LSpeC/tO3SI8g9X7n4F
4K2VGGS5AQ0ESCjfxxAEAKDFoVXqAAOI1LMetFiSqbBxc4cf813955ltLY88qJi8
1bJTWFQe1YFl4Kq/i71gkri9xlkNnERemipsyoHv/i2Pb1FeQvzLm6BnvZqRCqpQ
eRg4TOcaoQ/kFecBTBg4rGwVXG25Jjffr336LJ0/cO0LQGcZxRadAwv3l50XYubX
AAMHBACWK80MNxV9bVjO9RwY7DHfchETq7NtLVFwxjkdPacROfqQSEJ0i1c1eJ+Y
wPt22i9B3tT0g26wtqexKp0GAE6FP0IHrrw8QeI1BEkepvF3IPfVspJNJ89/0e3u
Wvkk7n6QtRbfYKp2M+V6WSh76TgtUdCxG7xNxsQXQYu5e+cK24hPBBgRAgAPBQJI
KN/HAhsMBQkDwmcAAAoJEHH+kikbmnHg/x0AoKRNFGP+WKUZxehMIzutH6lPWr78
AJ9K+Ng5s0mUCvPX1PmnDJG6uqkI4A==
=oRdN
-----END PGP PUBLIC KEY BLOCK-----

top

My Dellbuntu laptop: installing Kubuntu

KWTm KWTm writes  |  more than 7 years ago

Installing Kubuntu:
- the plan is to shrink the built-in Ubuntu partition on /dev/sda6 down to 6GB (my original plan was 4GB, but there's a frigg'n 4.3GB of files in there already! What did they load on there, some voice-recognition stuff?)
- expand the swap partition to 4GB (since there's 2GB of RAM in there)
- then add another 4GB partition to install Kubuntu
- then the remaining 130+ GB is for data

- running from the Kubuntu 7.04 Live DVD, QTParted is unable to resize the main 151GB partition! --can't move, can't resize; I can only choose to delete it if anything. The partition was not mounted, so not sure why I couldn't manipulate it.
- so we need some other tools for managing it, instead of QTParted. I bought PartitionExpert from Acronis (in 2003, it was $45, downloaded from web), which is able to handle Ext3 as well as ReiserFS partitions. At the time, Partition Magic (which was v6 at the time) was not able to do ReiserFS. Anyway, so in this step I was not able to move ahead with any FLOSS disk partition manager known to me. (Any suggestions for other partition managers I should try, in the future?)
- using Partition Expert: surprise! The 4.3GB of data turned out to be only about 1.7GB, taking up less room than I thought. I was able to shrink the gUbuntu partition (/dev/sda6) down to 4GB, create a new 4GB partition for Kubuntu (/dev/sda7), and make the remaining 135GB partition (/dev/sda8) for data.

(Here are partial results for "df -h", listing device, total size, used size, remaining size, %used, and mount point.) /dev/sda1 47M 876K 47M 2% /media/sda1 /dev/sda2 2.0G 693M 1.4G 34% /media/sda2 /dev/sda3 193M 21M 163M 12% /media/sda3 /dev/sda6 4.0G 1.7G 2.1G 45% /media/sda6 /dev/sda7 4.0G 2.8G 1000M 74% / /dev/sda8 135G 3.4G 132G 3% /media/sda8
(/dev/sda4 is not listed. That's the swap partition, which I increased to 4GB.)

- it took a long, LOOOooooo..ng time to do it, though. Using Partition Expert, I first clicked on the "big" partition (at the time it was /dev/sda6, 150GB), just to say, "I'll work with that partition". A window popped up: "analyzing partition", and then it froze. At least, it looked like it. Actually, it really was analyzing the partition, except it took 20 minutes! So, beware not to click on the wrong partition by accident.
- and then once I specified how I wanted the partitions set up, it took another 40 minutes or so. Not sure --it took so long that I lost track.

- Once everything was resized, Kubuntu installed smoothly.
- Previously, installing Kubuntu on my desktop boxes, I did very little manually; instead, I typed commands into a script and then ran the script, eventually accumulating a long record of my installation steps in a shell script. This paid off now as I copied and pasted large chunks of this script into a similar installation script for the Dellbuntu, and it took only about half a dozen steps (with big sections of nothing but "sudo apt-get --assume-yes install" lines) to reproduce my desktop configuration on the Dellbuntu.
- by installing the 915resolution package ("sudo apt-get install 915resolution"), I got to use the full resolution of 1280x800 \
- I previously worried about the screen being too small, but I guess with the full resolution the screen looks pretty big. I'm rather pleasantly surprised. My desktop has a 19" monitor capable of 1280x1024, but the Linux driver for the onboard graphics card can only drive it at 1024x768. I never realized till now how much screen size is dependent on resolution rather than just physical size.
- By the way, I chose the normal matte screen rather than the default Dell TrueLife(tm) Screen With Sharper Colours And Annoying Reflection. I'm glad! The screen is readable in a variety of lighting conditions.
- while on the subject of hardware, the keyboard feels nice. The one minor thing I have to get used to is the mousepad: occasionally, one of my thumbs accidentally touch it while I type, and the computer thinks I clicked on the mouse button.

Next step: installing Beryl! (Of course! That's the whole point of getting a laptop with Linux on it --to show off! :) )

top

Dellbuntu laptop arrived!

KWTm KWTm writes  |  more than 7 years ago

Today, the Dell laptop (Inspiron 1505n) with pre-installed Ubuntu arrived today. I will post my review blog as replies to my previous journal entry about Buying A Dellbuntu so that all the comments and threads can be organized into one place.

News flash: Nope, I won't be posting it to my previous journal entry because Slashdot has archived that entry and its replies, and no one can make changes. What the tutti fruitti!?? Okay, fine, I'll post the review as a reply to here.

top

Buying a Dellbuntu

KWTm KWTm writes  |  more than 7 years ago

I'm checking out the journal system on Slashdot. Hmm, it looks like I can enable and disable comments for individual journal entries. That's good.

Anyway, in this post about buying a Linux computer from Dell, I said I would post in my journal about how the purchase was going. Knowing me, I'll probably be too lazy to post much, but I figured that posting this in my journal would let others comment without having to post directly under the main thread.

Here's a copy of what I had posted:

I thought I should hang back and let others do the initial buying, to see how well this works out and whether the hardware crashes and burns. But if everyone did that, then nobody would buy because no one would want to be first. Since I've been looking forward to getting a Linux notebook, I think it should be okay for me to be one of the first "tryer-outers". Also, hopefully this venture of Dell's into Ubuntu will be high-profile enough that if I encounter any problems, I'll scream and shout that I'm going to post about my problems on Slashdot, and then Dell shall suffer the wrath of Slashdot!! and they'd be more willing to fix it.

In addition to the basic notebook at $599, I decided to upgrade the memory from 512MB to 2GB (+$200), since it's probably the most precious commodity around; if I try to upgrade later, say in 2 years, some new memory standard will probably have come out and I won't be able to find the proper chips.

I figured I'd upgrade the hard drive, too, from 80GB to 160GB. I had thought I would upgrade the 2.5" HDD myself, but it comes with a SATA hard drive, and I've only worked with PATA hard drives[1]. Anyway, that's another +$125 for the HDD upgrade.

My third upgrade is for the DVD burner. The original price comes with a CD burner/DVD-ROM drive, but I've always had problems with Linux and DVD burning --my Kubuntu box has the LITE-ON DVD DL burner, and so far I've had to power up our Win2k box to burn DVD's. For +$40, I'm happy to get the DVD DL burner, and I want to see if K3b will let me burn all 8GB+ onto a DL DVD. Would be sweet if I could.

The only thing I don't like is the screen size. I don't care about widescreen[2], and you can't directly compare diagonal screen sizes of 16:9 (widescreen) screens with 4:3 (conventional) sizes, so I converted. The diagonal of a 16:9 screen is 1.22 times as long as a 12:9 (that is, 4:3) screen for the same height, so I divided the 15.4" diagonal length of the widescreen by 1.22 to get 12.6". So I'm really getting a 12.6" screen, except it's wider. That's tiny. The ThinkPad that my work gives me is 15" (4:3 aspect, same screen height as 18.3" widescreen) and I don't think it's big enough. Well, at least the small screen size makes the laptop smaller and portable.

By the way, what the heck is "TrueLife (glossy)"? I have the option to have it or not have it for my screen, at the same price, but it sounds like a load of MarketSpeak.

So, anyway, here's my system, cut&pasted from the Dell page:

Intel® Pentium® dual-core proc T2080(1MB Cache/1.73GHz/533MHz FSB
Ubuntu Edition version 7.04
15.4 inch Wide Screen XGA Display with TrueLife(TM)(glossy)
2GB Shared Dual Channel DDR2 SDRAM at 533MHZ, 2 DIMM
160GB 5400 RPM SATA Hard Drive
8X CD/DVD Burner (DVD+/-RW) with double-layer DVD+R write capability

53 WHr 6-cell Lithium Ion Primary Battery
Intel PRO/Wireless 3945a/g

1Yr Ltd Warranty and Mail-In Service
Recycling Kit and Plant a Tree for Me

Intel® Graphics Media Accelerator 950
Integrated Audio
Intel Centrino Core Duo Processor

I'll probably sit on this till next week, and then make the purchase.
Any comments? Is this a good deal, or am I being foolish?

I'm experimenting with the Slashdot journal, so maybe I'll post stuff in my journal [slashdot.org] about how the purchase is going, and I think I can set it up so that people can post comments.

-----
[1] PATA notebook drives: It's not that I'm afraid of SATA drives; it's that I've been standardizing on PATA 2.5" drives because I have a number of 2.5" notebook enclosures that, for $25, turn the internal notebook HDD into an external USB HDD that fits into my shirt pocket.

[2] widescreen: Please don't give me that crap about "But if you're screen's not wide enough, you don't see the whole movie --it will be chopped off at the left and right sides!" Well, then, just shrink the movie! I don't see anyone ever saying, "You need a 4:3 screen, because your TV show will be chopped off at the top and bottom by a 16:9 screen!"

top

My OpenPGP key

KWTm KWTm writes  |  more than 8 years ago

-----BEGIN PGP PUBLIC KEY BLOCK-----
Version: GnuPG v1.2.4 (GNU/Linux)

mQGiBERe4zsRBACadO3RJcTnmyzjemGXqXuwbVJtSkijmHaB5nlSvn/rXpbk5H9l
GKK1Dog3V6x74Pj0AwTyaZLRpKT1pjk4MRv7zFnw2W3ZTxbnVduw7n+DnIRWZr38
2TRWUR96BlDwC6iUDwvAqosv+m5YrUvnuuLvOxTyfdt5OQWvHUOnfSnWiwCglnGq
GQtGa09nikCpQDi3l4e/lgMD/0e8MXWOYNZAMbSA1fQs8ZgpM1+Ll2r8/MZ8KGjM
brUhdcRx4gl62/9mq4as5R/1OURs2OvWkrSYfv/jdHRoZwiYAYsY0Jef7O58vqfK
OaOELG3mITh21fmcxrbCqO/0IgdlVC0kTWUNUBWAm1y1jE87SHly2AOIbOHUdwN4
Ads7A/9XWPvrT2Z5d7QWnT/45YqYRSJdyN8MuAuUPz580Xd4JYht4lR65SUB+yU/
kyKHnfZ0smg9MfKnbMcStTiqOGV1DTPsPClWtcI371QNbJxWuhFsStjJNT0yuCBC
vMs5lYV6UdFhcnI+T749VuWPd2foCNuXo0M/GV0yoI5W0GFMb7Q2S2EgV2FpIFRh
bSAoMjAwNiBNYXkgIkdudSBQRyIpIDxrYXdhaS1rYkB0YW1seWxpbi5uZXQ+iGEE
ExECACEFAkRe4zsFCQPCZwAGCwkIBwMCAxUCAwMWAgECHgECF4AACgkQB8iUW0ha
VSkqbQCeKecVPYM/L/P+A7oPg695Ne7mAJ8AniODa07BU4KgJ/xp8sKviQfOEt54
uQINBERe40QQCADdRZSrU/yAvsfIAf457k1xGETGsnZAPozzRe95KSqLMxAjeQ04
O4SxZYP6YuNEnE6J60VFfrvyv/uKxJtB3SRXYCwL1WG83LPcdEr6HMtRfyBH+fcA
sMTVuPKOfZqRcRJmKu9C1i1IjGgaT65bDJWHxHmWsnM3lfzIR4iOIaDDnQLwGogR
SYk7DG6jggoixi+tNRlssntl91iyxA8P5L9H08xih2bwXR3RGjwFBM1y/lBlPWZy /Vwf0FqUVas9GwVqvQytCTHnzigK1XUam2Q0NYg+7TiUn0NssJklcyMcD/oVTStd
lFCAYnRUzn/M3XB3Rwn4mp1kNAr0ldKP1ZJ3AAMFB/9LNRwhCaOFt6FI7WYdJrZr
1k+98RoktIL42Sm6TuMK2r4w/jtnby2gMMhOctnIdQsLgpjpYUJLQz9Q3K91Gw/k
8CwDrKJKzuTXgu9ds+HERx/3qMTVrWKXQ93XOBRDjd7iw2VqNKPQcc1/D+VXNIr4 /FReanrPvlrAjeTJC+Evk3HabfHgtmCj9HOrlj0FSanHjbunO4D0i+Bk4ZXPVsb2
vNPY14niiHnZFXefhXq2OUoy16fND0IZ3wYndslcjnar/bPvtTJ/KVeRMvxjGgQw
cNsxuxe2y5Iu5IAGpHvG/mnuGCMnjO4s2mTMfikVRTOQdKeTg9cocrVauEjq+jxG
iEwEGBECAAwFAkRe40QFCQPCZwAACgkQB8iUW0haVSlEjgCeN0qzjK49n2q1Wr+8
cUhtJLj+UG0An22+XOPL1TIwf3LZfyc6y1pTSp45
=nLy2
-----END PGP PUBLIC KEY BLOCK-----

Slashdot Login

Need an Account?

Forgot your password?