reading csv file in mathematica

When Mathematica reads a flatfile, it evaluates each field as a mathematica expression. I sometimes just want to read the entire file in as strings then perform some specific type conversion on columns. Here's a short way to import data is string fields.

The idea is to provide a shell of the algorithm and then extensive customization through functions. This is a very functional approach to creating functions. However, the package is still composed of a giant function though versus a more clever, parsing state machine.

You have options to customize line splitting, field/attribute processing, field level transformations as well a collection mechanism that allows you to collect specific input record for later use. For example, you can save the header line. The returned value is not just a data matrix, but an association with various import statistics and other collected information.

You can find the package here in a gist.




Comments

Post a Comment

Popular posts from this blog

quick note on scala.js, react hooks, monix, auth

quick note on scala.js, react hooks, monix, auth Reactjs hooks finally came out of preview in a recent version of React. scalajs-reaction has had support for hooks since they were in preview. React hooks seem like the hot new thing but for me the question is, do hooks help improve your code by improving maintainability, time to market, TCO, etc.? Let’s take a look using authentication as an example and employ monix , a reactive observable framework that also builds on cats . We will assume we are building a SPA. Does it improve your code and make your application & development experience better? Auth Auth processing can be complex. Many react auth examples you find in blogs like to show how to use “react-routing” and “auth” as a switch that determines which routes are protected or unprotected. Protected routes show application specific data. Unprotected routes show login content . Based on storing a “User” object in the browser session/local storage, th
Read more

zio environment and modules pattern: zio, scala.js, react, query management

zio environment and modules pattern: zio, scala.js, react, query management This blog covers a zio-based fetch design for scala.js, react web applications. It heavily uses the zio “environment” feature to help motivate a point of view on the modules pattern in effect systems. At the end, I summarize some thoughts on ergonomics and complexity. This blog assumes a basic knowledge of zio and react. scala.js is just like scala so you mostly already know scala.js. This blog is a follow-on to previous blogs: https://appddeevvmeanderings.blogspot.com/2019/10/scalajs-react-fetcher-hook_48.html and https://appddeevvmeanderings.blogspot.com/2019/11/scalajs-and-react-suspense.html . I have been keeping an eye on and culling ideas from typscript/javascript fetching libraries that address query management concerns. The formulation below incorporates some of those concerns but does not try to comprehensively duplicate well established javascript libraries. I only care about
Read more

user experience, scala.js, cats-effect, IO

user experience, scala.js, cats-effect, IO This is not a complicated snippet but I wanted to ensure I remembered it. When creating a UI, say react for browsers, you will run into the “fetch” data problem. When you fetch data, you typically have a component that performs the fetch and acts as a cache for the result. The fetch component renders a child and passes along the data that was fetched (or the fetch state, etc.). Even with the upcoming react suspense mechanism that will make this a smoother user experience, you may have a flash of a “loading” indicator that shows for a moment before the content is fetched from the server. If the flash is too fast, it is visually disruptive. A shimmer mechanism for the UI my also not be the answer. Similarly, if the fetch request takes too long, you will want it to time-out and show an error message that the data could not be fetched or whatever is appropriate for feedback in your application. Generally, I use cats-effect
Read more