Day 14 — Goddamned Tests and Registration/Authentication (go to project overview)
So far, much time spent getting the search test working. I don’t want to make an actual call to Twitter (because it’s not free, mostly, but) because it’s against the rules of a test. So, I have some twitter data in a file. I produced my file incorrectly and have been struggling with that for longer than I want to admit.
In the end, to get it to work, finally, I just took the Twitter response.content and wrote it to a file like this:
Sometimes we think we are doing A when we are really doing B
f = open('common/test_data/test_response2.json', 'wb')f.write(response.content)Kinda painful when you realize all of the stuff you have been doing in code really has nothing to do with anything. I include this, here, though, because people who need to test this app will need it and I’ll probably put it in the readme file and I probably won’t really have any idea what I did when I’m writing the readme. The steps should be easy, so I’m deliberately not documenting most of them as I’m working. If I can’t come up with them when I write the readme, they’re probably not simple enough.
Doing the final test of tests and basic functionality, then moving onto registration/authentication. Plenty of Twitter gets remaining for the month. I need to get the authentication done and use as many of those tweets as I can before I get the new October tweet batch.
No registration. I don’t want strangers using up my tweet allotment!
My one last test took awhile because I’m trying to export a csv file depending upon the request and that didn’t work. I did discover, though, that if Twitter returns an error—even if it’s an error due to a poorly formatted request—the request is not counted against your allotment. That’s comforting!
Curse you secret key!
Ha…I just discovered that I DID commit my django secret key although it was in comments. It will have to be generated, anyway, before I make the repository public just because anyone that wants to use it will need to generate a secret key, so I’ll need to be able to provide instructions. Still, I thought I was being so careful and I botched it.
Also, I’m seeing that the PHP code I built way back when reflects my confusion on the max_results field. I thought it was max requests…as in don’t do next until there’s no more data. The field Twitter allows us to pass is max results (which is 100 per request for sandbox and 500 per request for premium). But, since the subscription max results is always max requests for the subscription level * max results for the subscription level, It seems like I would always want max results to be set at the max for the subscription level so I get the most from my calls. As I’m writing this, I’m wondering if I didn’t just introduce this confusion.
This will be easy with a Javascript Library. I’ll wait to fix it.
Onto errors. I’m downloading the file using location.replace(my_view_url). This works great unless there are errors in the view. I’m not sure how to fix this. At some point, hopefully sooner than later, I’m planning to add React with Django Rest Api. That will, no doubt, correct this problem. I’m going to move on to adding the Twitter Counts calls and come back to this later. I’m seriously having a “I am cheap” moment. In order to use the counts endpoint, I have to upgrade to premium. I do not want to spend money given my current employment situation. But, I can’t test any code, unless I do the upgrade. I gotta do it. Thank you, I’m glad we had this talk.
Tests are good!
Also, before continuing, I ran all tests. Of course some are broken. The main problem is that I decided to get rid of some saved search stuff that I thought would be cool, but now I don’t think so. Also, now that I’ve added the login stuff, the cool generic view tester didn’t work with AnonymousUser. Otherwise, I’m ready to continue.
The counts changes were fairly easy and straight forward. I added an additional response file to return from the endpoint mock for counts. I have accidentally committed my mock test .json file to Github. I don’t know if this is ok or not. The data belongs to Twitter. Since I’ve done it, I’m going to think about it later. It IS very helpful to me to see actual data when I look at code, so it would be nice for people to have…
Next, I’m going to add some fields to the Researcher model to keep track of previous queries.