python - Pandas read_fwf: specify dtype -

-

i reading in huge fixed width text file in chunks , export data csv. because pandas.read_fwf not allow specify dtypes, wondering other way there exists force columns strings. reason pandas infers columns float though not , not want .0 within column.

using data[column] = data[column].astype(str) not not rid of decimals. converting columns of float64 dtype int doesn't work either since nas cannot converted. ideas?

here's snippet of code:

dat = pd.read_fwf(file_to_read,colspecs=cols,header=none,chunksize=100000,names=header) #first chunk data.info() int64index: 100000 entries, 0 99999 columns: 562 entries, dtypes: float64(405), int64(4), object(153) memory usage: 429.5+ mb  column in data.columns:     if data[column].dtype == 'float64':         data[column] = data[column].astype(int)     else:         pass

i str().replace('.0',''), want find easier way iterating through column takes lot of time.

the converter parameter can used preserve data strings since pd.read_fwf not try guess dtype if converter specified:

import pandas pd try:     # python2     cstringio import stringio  except importerror:     # python3     io import stringio  content = '''\ 1.0    2    3.0    4    b 5      x    c m      y    d ''' header = ['foo', 'bar', 'baz']  df in pd.read_fwf(stringio(content), header=none, chunksize=2, names=header,                       converters={h:str h in header}):     print(df) df.info()

yields

   foo bar baz 0  1.0   2   1  3.0   4   b    foo bar baz 0   5   x   c 1   m   y   d  <class 'pandas.core.frame.dataframe'> rangeindex: 2 entries, 0 1 data columns (total 3 columns): foo    2 non-null object bar    2 non-null object baz    2 non-null object dtypes: object(3) memory usage: 120.0+ bytes

Comments

Post a Comment

Popular posts from this blog

java - nested exception is org.hibernate.exception.SQLGrammarException: could not extract ResultSet Hibernate+SpringMVC -

-
i trying fetch records database , having above error this classs appropriate getters , setters public class user { private int id; private string username; private string password; private string email; i have create database , inserted data below configuration <bean id="datasource" class="org.apache.commons.dbcp2.basicdatasource" destroy-method="close"> <property name="driverclassname" value="com.mysql.jdbc.driver"/> <property name="url" value="jdbc:mysql://localhost:3306/usersdb"/> <property name="username" value="root"/> <property name="password" value=""/> </bean> this complete stacktrace severe: servlet.service() servlet [appservlet] in context path [/spring] threw exception [request processing failed; nested exception org.hibernate.exception.sqlgrammarexception: not ...
Read more

sql - Postgresql tables exists, but getting "relation does not exist" when querying -

-
Image
i have postgresql db number of tables. if query: select column_name information_schema.columns table_name="my_table"; i list of columns returned properly. however, when query: select * "my_table"; i error: (programmingerror) relation "my_table" not exist 'select *\n "my_table"\n' {} any thoughts on why can columns, can't query table? goal able query table. you have include schema if isnt public one select * <schema>."my_table" or can change default schema show search_path; set search_path my_schema; check table schema here select * information_schema.columns for example if table on default schema public both works ok select * parroquias_region select * public.parroquias_region but sectors need specify schema select * map_update.sectores_point
Read more

asp.net mvc - breakpoint on javascript in CSHTML? -

-
i have .cshtml file has javascript tag. within script tag have few lines of javascript. first line of javascript refers property in @viewbag. other javascript lines plain script without mvc related references. observation: can place breakpoints (visual studio 2015) on each of these javascript lines. however, breakpoints has white dot in center of breakpoint symbol (red circle) except line @viewbag reference (this particular line not have white dot in center). it seems breakpoint without white dot hit during run-time. question: can explain going on here? white dot mean? why plain red breakpoint line line hitting breakpoint? the visual studio debugger expecting debug actual server-side code within .cshtml file opposed client-side javascript within it. iirc, visual studio allow debug javascript code expected in internet explorer (and possibly edge), other browsers want use either third-party tool or developer tools (f12) within browser. an easy approach take advant...
Read more